ISTQB Foundations of Software Testing

04 –
Test
Desig
n
Tech Rendering
nique services
s
for:
ISTQB Foundations of Software testing

– Fundamentals of testing
Timisoara Engineering Centre
Speaker: Laura Budulan
1 BeeSpeed Timisoara Engineering Centre, 2017

Agenda Rendering
services
for:
1. What testing is and what testing does?

2. Why is testing necessary?
3. General testing principles
4. Test process – Activities and Tasks
5. Test process – Test Work Products
6. Test process – Traceability
7. The psychology of testing
8. Check of understanding

Agenda
Rendering
services
for:

3. Seven testing principles

What testing is and what testing does?
Rendering
services
for:
– Testing Activities
Testing is an activity to reduce risk and improve quality by finding defects.
The process consisting of all life cycle activities, both static and dynamic, concerned with planning, preparation and
evaluation of products and related work products to determine that they satisfy specified requirements, to determine
that they are fit for purpose and detect defects.
 Testing and debugging

- Debugging is the process to identify cause of bugs in code and undertake corrections
- Testing is a systematic exploration of a component or system to find and report defects
- Both are needed for achieving a good quality

Rendering

services
for:
– Testing Activities
 Static testing and Dynamic testing
- Static testing is where the code is not exercised (e.g Reviews)
- During Dynamic Testing the program under test is exercised with some test data
 Testing as a process
-Testing means not just test execution but, design, record, checking for completion
- We design test process to ensure not to miss critical steps and do things in the right order

Rendering

services
for:
– Objectives of testing
The main goals of testing:

 To evaluate work products such as requirements, user stories, design and code
 Find failures and defects
 Prevent defects
 To validate whether the test object is complete and works as the users and other stakeholders
expect
 Build confidence in the level of quality of the test object
 Provide relevant /sufficient information to the stakeholders to allow them to make informed decisions
 To ensure all requirements are fulfilled
 Reduce the level of risk of inadequate software quality
 Increase the product quality
 To assure that required standards and legal requirements are met
Rendering

services
for:
– Objectives of testing
Different viewpoints and objectives:
 Component testing – find as many defects as possible/ increase code coverage

 Acceptance testing – confirm that the system works as specified and that the quality is good
enough/ give information to stakeholders about the risk of releasing the system at a given time.
 Testing metrics gathering – provide information to the project manager about the product quality and
the risks involved
 Design tests early and review requirements – help prevent defects

Agenda
Rendering
services
for:


Why is Testing Necessary?
Rendering
services
for:
- Testing’s Contribution to Success
 Examples of Testers contribution within the software development lifecycle:
 Requirement reviews – to review the specifications for completeness and correctness, ensure that they are
testable; identification of requirements defects -> reduce the level of risk of functionality defects
 System Design – to improve interfaces testability and usability. Increase each party understanding of the
design -> reduce the risk of design defects
 Programming – to review the code and assess structural flaws. Increase each party understanding of the
code and how to test it. -> reduce the level of risk of code defects
 Testing prior to release – detect failures and support removal of defects(eg. Debugging) -> increase the
likelihood of meeting stakeholder needs and requirements.

Rendering
services
for:
- Why do software have bugs and defects?
 An incorrect software can harm everything

 People (in an aircraft crash, or a hospital life support system fails)
 Companies (incorrect billing, tax, lose money)
 The environment (releasing chemicals or radiation)
 The effect of software failures lead to

 Loss of business reputation
 Loss of time
 Loss of money
 Injury (or even death)

Rendering
services
for:
- Why do software have bugs and defects?

 Software-defects are frequent problems
 Examples:
■ ESA – Ariane 5
■ NASA – Mars Climate Orbiter
– Lost its line
– Lost in space
– Reused code
– Data mistake: metric vs. English measures
■ Online British taxation

– The previous person’s data was visible ■ Electronic invoicing system(USA)
– $7000000 electronic meter in place of $10000
■ British „top 10 criminals” webpage – New software (compatible with Y2K)
– Server shutdown because of overloading
– Sudden huge interest ■ Telecommunication (USA)
– Free long-distance calls
■ Online bookshop
– Negative order results in refund ■ Almost nuclear war
– Administrative function available to customers – False alarm at a Soviet rocket station

Rendering
services
for:
-Error, Defect, Failure
 Who can make an error or mistake?

-Everyone who works
-Especially under pressure
Error / Mistake Defect / Bug Failure

- Misunderstandings during - Defect is the effect of an error - Deviation of the component or
meetings - A wrong picture in designers mind system from its expected delivery,
- Mistype during writing a document about the system service or result.
or a code snippet - It is carried by documentations, code - Can be caused by a defect or by
- Error happens at the moment of or people mind environmental conditions (eg.
the activities Electromagnetic fields)
An error that leads to a defect in a work product can lead to another error that can
lead to a defect in the code.
An Incident is any event occurring that requires investigation.
Rendering
services
for:
-False positive – False negative
 False POSITIVE:
- Reporting a defect that does not really exist
- Also sometimes called: false – fail of a Test Case
 False NEGATIVE:
- Failing to identify a defect that is there
- Also sometimes called: false-pass of a Test Case
- It is the worse

Rendering
services
for:
Defects, Root Causes and Effects
 The root causes of defects are the earliest actions or conditions that contributed to creating the
defects.
 Defects can be analyzed to identify their root causes, so as to reduce the occurrence of similar
defects in the future.
 Root cause analysis can lead to process improvements that prevent a significant number of future
defects from being introduced.

Rendering
services
for:
- Testing Against Errors
 If we wish to influence errors

 We need to begin with testing as soon as we begin making errors
 Right at the beginning of the development process
 We need to continue testing until we are confident that there will be no serious failures
 Right at the end of development process
 Through the whole development Software Development Life Cycle (SDLC)

Rendering
services
for:
- Consequence of Mistakes
 Victims of the failure:
 humans
 an organization
 the environment
 Results of the failure:

 financial loss
 loss of time
 reputation degradation
 injury
 death
 It is necessary to start bug searching ASAP:

 The testing process must be started together with the development process, because we can make mistakes from
the beginning of the development
Why is Testing Necessary? Rendering
Why can defects be left in the system? services
for:
 Because of inadequate testing!

 Testing for a short time - time pressure
 Bad types of tests are used
 A perfect development does not exist
 The development is a human creative activity which carries a mistake opportunity
 Inexperienced or insufficiently skilled project participants
 Miscommunication between project participants
 Misunderstandings about intra and inter-system interfaces
 New, unfamiliar technologies
 „We can be rather good if we aim for perfection”

 More and better tests seem to be reasonable, but it is not that simple.

Rendering
services
for:
- Numbers, Numbers, Numbers
$ 100,000 >> Cost of 1 hour unscheduled halt
10,000 x >> Cost of live environment correction
80% >> Unplanned halts could be prevented with

appropriate testing
75% >> Application is introduced without testing

Rendering
services
for:
- Can we test everything?

 The vison is to test all the things
 Exhaustive Testing of complex system is simply not possible
Exhaustive testing is a test approach in which all possible data combinations are used. This includes implicit data
combinations present in the state of the software / data at the start of testing.
- There is a system with 20 independent inputs, each has 4 possible values

- To exhaustive test this we would need 4 20 = ~1012 Test cases
- Each test takes 1 minute
- To run all the Test Cases for one person would take more than 2 million years
 Furthermore, for testing everything we would need to start testing from the beginning after every little
modification
 The aim is the better efficiency of testing

Rendering
services
for:
- Testing and Risks
 Risk is inherit in all software development

 Uncertainties become more significant as the system Greater
complexity grows and the implications of failure increase possibility of
More failures
complex
systems Greater
impact of
 Testing depends on the risk failures
- Video games
- Flight control systems
- Automotive braking
! Grater risk implies more and better testing

Rendering
services
for:
- Testing and Quality
 Many people and organizations are confused about the difference between quality assurance (QA), quality control
(QC), and testing
and Quality
 Quality Assurance: A set of activities designed to ensure the adherence to proper processes, in order to provide
confidence that appropriate levels of quality will be achieved.
 Quality Control: A set of activities, including testing, designed to support the achievement of appropriate levels of
quality.
 Quality Management: The concept that links QA and QC. Includes all activities that direct and control an
organization with regard to quality.
!Quality assurance supports proper testing!
!Testing contributes to the achievement of quality!

Rendering
services
for:
 What is quality?
 When can we say that our software has good quality?
 Quality measurements:
- Adaptability -> how easily software can be modified to meet new requirements
- Maintainability -> how easy it is for the developers to maintain the application and how quickly maintenance
changes can be made
- Modularity -> how much of a system or computer program is composed of discrete components and a change
to one component has minimal impact on another component
- Correctness -> how accurately the software perform the functions defined in the specifications
- Reliability -> the time or transactions processed between failures in the software
- Efficiency -> how well a component performs its designated functions using minimal resources
- Usability -> the ease of use of the software by the intended users
- Reusability -> how easy it is to re-use elements of the solutions in other identical solutions
- Testability -> how easy is to test the applications, clear unambiguous requirements
- Legal requirements

Rendering
services
for:
 How can we achieve better quality in our software solutions?
 Quality against resources

- If we need to deliver a software faster, it will usually cost more – with the same quality
- Time, Money, and Quality affects one another and also influences the features that are included or not in the
delivered software
quality
Efficiently and/or effectively
testing? Features
Cheaper and/or more safe

testing? ?
money time
Rendering
services
for:
- How Much Testing is Enough?
 What we know about testing so far?

- We cannot test everything
- Every system is subject to risk of one kind
- There is a level of quality that is acceptable for a given system
 What should we need?

- Prioritization, to be able to run most important tests first
- Completion Criteria, to give an objective test of whether it is safe to stop testing
- How much of the Software Under Test is to be tested?
- What levels of defects can be tolerated in the product?

Rendering
services
for:
Keeping the Software Under Control – Goal
 Best achievable quality using the given resources

■ „Less critical’’ software
or
 Attaining a given quality level using as few resources as possible
■ „Critical’’ software

Agenda
Rendering
services
for:


General testing principles
Rendering
services
for:
 Testing shows the presence of bugs

- Testing doesn’t prove that the software is error free
 Exhaustive Testing
- Risks and Priorities
 Early Testing
 Defect Clustering
- System complexity
- Volatile code
- Effects of change upon change
- Development staff experience or inexperience

General testing principles
Rendering
services
for:
 “Test Pesticide” paradox

- The phenomenon that the more you test software, the more immune it becomes to your tests - just as insects
eventually build up resistance and the pesticide no longer works.
 Testing is context dependent
 Absence of errors fallacy

- It is possible that software which is 99% bug-free is still unusable. This can be the case if the system is tested
thoroughly for the wrong requirement. Software testing is not mere finding defects, but also to check that software
addresses the business needs. Absence of Error is a Fallacy i.e. Finding and fixing defects does not help if the system
build is unusable and does not fulfill the user's needs & requirements.

Agenda
Rendering
services
for:


Test Process
Rendering
services
for:
- Test process in Context
Factors that influence the testing process in an organization include:
 SDLC model being used
 Test levels and types being considered
 Product and Project risks
 Business domain
 Operational constraints (e.g. budget, resources, timescale, complexity, contractual requirements)
 Organizational policies and practices
 Required standards

Test Process
Rendering
services
for:
- Test Activities and Tasks
 The most visible activity is running one or more tests: Test execution
 Before that we also have to prepare for running tests: Planning
 During the work we have to be sure that everything go as we planned before: Monitoring and Control
 We have to analyze the test basis: Test Analyzing
 We have to design the test cases to be run: Test Design
 We have to implement the test cases to be run: Test implementation
 After all we have to close the testing phase: Test completion

Fundamentals of testing Rendering
services
for:
Fundamental Test Process

Test
Test Planning Test analysis Test
implementation
and Control and design completion
and execution
 Test planning
- Defining the scope and objectives of testing and identifying risks
- Determining the test approach (techniques, test items, coverage, identifying and interfacing the teams involved in
testing)
- Determining the required test resources (people, test environment, PCs)
- Implementing the test policy and / or the test strategy
- Scheduling test analysis and design tasks
- Scheduling test implementation, execution and evaluation
- Determining entry and exit criteria (see chapter 5.2)
- Testing a system’s configuration data shall be considered during test planning, if such data is part of a system
 Test monitoring and control

- Measuring and analyzing results
- Comparing expected and actual progress, test coverage and exit criteria(see chapter 5.2) against the test plan
- Making corrections/actions in order to meet the objectives
services
for:

Test
Test Planning and Test analysis Test
implementation
Control and design completion
and execution
 Test analysis – what to test
- Review the test basis (such as requirements, software integrity level (risk level), risk analysis reports, architecture,
design, interface specifications)
- Evaluating testability of the test basis and test items
-
TEST BASIS: all documents from which the requirements of a component or system can be inferred. The
documentation on which the test cases are based. If a document can be amended only by way of formal amendment
procedure, then the basis is called a frozen test basis.
- Identifying features and set of features to be tested

- Identifying and prioritizing test conditions based on analysis of test items, the specification, behavior and structure
- Capturing bi-directional traceability test basis and test conditions

services
for:

Test
implementation
and execution
 Test design – how to test
- Designing and prioritizing test cases and set of test cases

- Identifying necessary test data
- Designing the test environment, infrastructure and tools
- Capturing bi-directional traceability between test basis, test conditions, test cases and test procedures

services
for:

Test
Test Planning and Test analysis and
Control design implementation Test completion
and execution
 Test implementation
- Developing, implementing and prioritizing Test Cases
- Developing and prioritizing Test Procedures and, potentially, creating automated test scripts.
- Creating Test Suites from the Test Procedures for effective test execution
Test suite: A set of several test cases for a component or system under test, where the post conditions of one test is
often used as the precondition for the next one.
- Building the environment and verifying that everything has been set up correctly
- Preparing test data and ensuring it is properly loaded in the test environment
- Verifying and bi-directional traceability

services
for:

Test
and execution
 Test execution
- Executing Test Procedures either manually or by using test execution tools, according to the planned sequence
- Recording the IDs and versions of the test item or test object, test tool and testware
- Logging the outcome of test execution (Pass/Failed) and recording the identities and versions of the Software Under
Test, test tools.
- Comparing actual results with expected results
- Reporting discrepancies as incidents and analyzing them in order to establish their root cause
- Repeating test activities as a result of action taken for each discrepancy or part of the planned testing(e.g.
confirmation and/or regression testing)
- Verifying and bi-directional traceability

services
for:

Test
Test Planning and Test analysis and Test
implementation
Control design completion
and execution
 Test closure activities
- Checking whether all defects reports are closed, entering change request or product backlog items for any
unresolved defect
- Creating a test summary report
- Finalizing and archiving tests data, the test environment, test infrastructure and other testware for later reuse
- Handover of tests to the maintenance organization
- Analyzing lessons learned from the complete test activities to determine changes needed for future iterations,
releases and projects
- Using the gathered information for the improvement of test process maturity

Agenda
Rendering
services
for:


Test Process
Rendering
services
for:
- Test Work Products - 2018

Test
Test Planning Test analysis Test
implementation
and Control and design completion
and execution
 Test planning
- Test plans, which include information about:
- Test basis
- Traceability
- Entry/Exit criteria
 Test monitoring and control

- Various types of reports
- Test progress reports
- Test summary reports

Test Process
Rendering
services
for:

Test
implementation
and execution
 Test analysis
- Test conditions – defined, prioritized and traceable to the test basis it covers
- Test charters – specific to exploratory testing
- Found defects in test basis
 Test design
- Test cases and set of test cases (high level)
- Design and identification of:
- Test data needed
- Test environment
- Infrastructure and tools

Test Process
Rendering
services
for:
- Test Activities and Tasks - 2018

Test
and execution
 Test implementation
- Test procedures and their sequence
- Test suites
- A test suite schedule
 Test execution
- Status of individual test cases or test procedures (e.g. pass, failed, skipped)
- Defect reports
- Documentation about which test item, test object, test tools and testware were involved in testing

Test Process
Rendering
services
for:

Test
Test Planning and Test analysis and Test
implementation
Control design completion
and execution
 Test completion
- Test summary reports

- Action items for improvement
- Change request items
- Product backlog items
- Finalized testware

Agenda
Rendering
services
for:


Test Process
Rendering
services
for:
- Traceability test basis – test work products
A good traceability supports:
 Evaluation of test coverage
 Analyzing the impact of changes
 Meeting IT governance criteria
 Improving the understandability of test progress reports and test summary reports
 Relating the technical aspects of testing to stakeholder in terms that they can understand
 Providing information to asses product quality

Agenda
Rendering
services
for:


The psychology of testing
Rendering
services
for:
– Participants
Very different people can be involved in software testing
Developers
Professional testers
Specialists
Users

The psychology of testing Rendering
services
for:
– Testers against Developers
Developers Testers
Developers want to prove Testers want to prove that

their code does work the code does not work
Often it is a subordination relation

Rendering
services
for:
– Independence
Test Independence - Those who wrote the software under test

testing should be
done by different
people than the Another member of the same development team
creation of the code
Members from a different organizational group (e.g. an independent team)
Members from a different organization of company (e.g. outsourcing to another company
Test Independence - Software Company

Companies
Software Company + Client
Software Company + Client + Independent Test Company

Rendering
services
for:
– Finding a Defect
 Tester: Proud
 Developer: Nervous
 Project Manager: Nervous
 Client: Nervous

Rendering
services
for:
– Defect Reporting
 Keep the focus on delivering a quality product
 Results should be presented in a non-personal way
 Attempt to understand how others feel
 At the end of discussions, confirm that you have both understood and been understood

Agenda
Rendering
services
for:


Agenda Rendering
services
for:
Check of understanding

04 –
Test
Desig
n
Tech Rendering
nique services
s
for:

Life Cycles

Agenda Rendering
services
for:
1. Software Development Models

2. Test Levels
3. Test Types
4. Maintenance Testing

Rendering
Agenda services
for:

2. Test Levels
3. Test Types

Software Development Models
Rendering
services
for:
 Testing does not exist in isolation

 Test activities are related to software development activities
 Different development model types

 Sequential development models
 Iterative and incremental development models
What is involved in a development life cycle for a software product?
Initial requirement from the customer
Expanding it for the code production
Writing the code
Testing de product
Ready for release

Rendering
services
for:
 Characteristics of good testing is any software development life cycle
 For every development activity, there is a corresponding test activity

 Each test level has test objectives specific to that level
 Test analysis and design for a given test level begin during the correspondent development activity
 Testers participate in discussions to define and refine requirements and design, and are involved in reviewing
work products

Rendering
services
for:
– Sequential models
 Waterfall model
 Shows the steps in sequence

 Each work-product or activity is completed before moving
on the next
 Testing is carried out once the code has been fully
developed
 Testing serves as quality check
 After test completion the product can be released into the
live environment

Rendering
services
for:
 V-model
 A process is needed which assures quality throughout the development life cycle
 At every stage a check should be made that the work product meets its objectives for that stage
 Characteristics of the V-model:

- A more complex, sequential development mode
- Extension of waterfall model
- In practice, a V-model may have more, fewer or different levels of development and testing,
depending on the project and the software product (e.g. system integration testing)
- Defects can be identified as early as possible in the life cycle
- Each stage must be completed before the next one can be started

Rendering
services
for:
 V-model

Rendering
services
for:
– Iterative Development Models
 Iterative development is the process of establishing requirements, designing, building and testing a system, done as a series
of shorter development cycles
 This process is repeated until a fully working system is produced
 The requirement do not need to be fully defined before coding can start
 The lack of formal documentation makes it difficult to test
 Developer changes many times are not formally recorded
 The resulting system produced by iteration may be tested as several test levels during each iteration
 Regression testing needed
 Forms of iterative development include

 Prototyping
 Rapid Application Development (RAD)
 Agile Software Development, Extreme Programming (XP)
 Rational Unified Process (RUP)
Rendering
services
for:
– Prototyping
 Prototyping is an iterative development model

 Proof-of-concept prototype or pilot
 Advantages
- Cost effective
- Increases system development speed
- Gives users an idea what the final system looks like
- Assists to identify any problems with the efficacy of earlier design, requirements-analysis and coding activities
- Helps to refine the potential risks associated with the delivery of the system being developed
- User interaction available in during development cycle of prototype
 Disadvantages
- Producer might produce a system inadequate for overall organization needs
- User can get too involved whereas the program can not be to a high standard
- Structure of system can be damaged since many changes could be made
- Producer might get to attached to it (might cause legal involvement)
- Not suitable for large applications

Software Development Models Rendering
services
– Rapid Application Development for:
RAD is an iterative
development model
Minimal planning (interleaved with writing the software itself)

Developer first builds preliminary data models and business process models of the business requirements
Analysts and users verify the requirements and formally refine the data and process model
Advantages
Promotes strong collaborative atmosphere

Dynamic gathering of requirements
Business owner actively participates in prototyping, writing test cases and performing unit testing
Disadvantages
Dependency on strong cohesive teams and individual commitment to the project
Success depends on disciplined developers and their exceptional technical skills
Decision making relies on the feature functionality team
Communal decision making process with lesser degree of centralized PM and engineering authority

Rendering
services
for:
– Agile Software Development
Agile Software Development is an iterative development model

 Requirements and solutions evolve through collaboration between self-organizing cross-functional teams
 Break tasks into small increments with minimal planning
 Iterations and short time frame (“timeboxes”) that typically last from one to four weeks
 Stakeholders produce documentation as required
 An iteration may not add enough functionality to warrant a market release, but the goal is to have an available
release (with minimal bugs) at the end each iteration
 Emphasize face-to-face communication over written documents
 Team size is typically small (5-9 people)
 Each agile team will contain a customer representative
 At end of each iteration, stakeholders and the customer representative review progress and re-evaluate priorities

Rendering
services
for:
– Extreme Programming
 XP is an iterative development model
 The beneficial elements of traditional software engineering practices are take to “extreme” levels, on the theory that if some is
good, more is better
 Programming in pairs or doing extensive code review

 Unit testing of all code
 Avoiding programming of feature until they are actually needed
 Flat management structure
 Simplicity and clarity in code
 Expecting changes in the customer’s requirements as time passes and the problem is better understood
 Lowers the cost of changes through quick spirals of new requirements
 Frequent communication with the customer and among programmers
 Most of the design activity takes place inclemently and on the fly

Rendering
services
for:
– Rational Unified Process
 RUP is an iterative development model
• Not a single concrete perspective process, but rather an adaptable process framework intended to be tailored by the
development organizations and software project teams that will select the elements of the process that are
appropriate for their needs
• RUP Is based on a set of building blocks
 Phases: INCEPTION – ELABORATION – CONSTRUCTION – TRANSITION

 Engineering disciplines
- Business modeling
- Requirements
- Analysis and design
- Implementation
- Test
- Deployment
 Supporting disciplines
- Configuration and change management
- Project management
- Environment
Life Cycles Rendering
services
for:
Software Development Models – Rational Unified Process

Agenda Rendering
services
for:

2. Test Levels
3. Test Types

Test Levels
Rendering
services
for:
Component test Integration test System test Acceptance test
• For both types of development testing plays a significant role

• Testing helps to ensure that the work-products are being developed in the right way (Verification)
• Did we build the product right?
Verification checks that the work-product meets the requirements set out for it.
• Testing helps to ensure that the work-products will meet the user needs (Validation)
• Did we build the right product?
Validation ensuring that the behavior of the work-product matches the customer needs as define for
the project.
• Characteristics of good testing across the development life cycle
• Each work-product is tested (V-model)
• Early test design (V-model)
• Testers are involved in reviewing requirements before they are released (V-model)

Test Levels
Rendering
services
for:
– Component test
 Code has been written before testing of the code can be started
 Generally code is written in parts or units
 The units are usually constructed in isolation for integration at a later stage
 Also known as unit or module testing
 Objectives
 Reducing risk
 Verifying the behavior of the component
 Building confidence in the component’s quality
 Finding defects in the component
 Preventing defects in higher test levels

Test Levels
Rendering
services
for:
– Component test
 Test basis:
 Component specifications
 Detailed design
 Code
 Data model
 Typical tests objects:
 Components, units or modules
 Code and data structure
 Classes
 Database modules

Test Levels
Rendering
services
for:
– Component test
 Typical defects and failures
 Incorrect functionalities
 Data flow problems
 Incorrect code and logic
 Specific approaches and responsibilities
 Defects found and fixed during unit testing are often not recorded
 One approach to component testing is to prepare and automate tests cases before coding (TDD = test driven development)
 This approach is highly iterative and is based on cycles of developing test cases, then building and integrating small pieces of code, and
executing the component tests correcting any issues and iterating until they pass
 Usually performed by the developer who wrote the code

Test Levels
Rendering
services
for:
– Integration test
 The purpose is to expose defects in the interfaces and in the interactions between integrated components or systems
 Integration test types
- Component integration test
Component integration test focuses on the interactions between software components and is done
after component (unit) testing.
- System integration tests
System integration test focuses on the interactions between different systems or between hardware
and software and maybe done after system testing of each individual system.

Test Levels
Rendering
services
for:
 Objectives
 Reducing risk
 Verifying the behavior of the interfaces
 Building confidence in the quality of the interfaces
 Finding defects
 Preventing defects in higher test levels
 Integration strategies
 Big-bang strategy – All units are linked at once resulting in a complete system
 Incremental strategy – Based on system architecture (top-down of bottom-up), functional tasks, transaction sequences
or other aspects as well
 In order to ease fault isolation and defects early, integration should normally be incremental rather than “big bang”

Test Levels
Rendering
services
for:
 Test basis:
 Software and system design
 Sequence diagram
 Interface and communication protocol specifications
 Architecture at component or system level
 Workflows
 Use cases
 External interface definitions
 Typical test objects:

 Sub-systems and database
 Infrastructure
 Interfaces
 System configuration and Configuration data
 API’s and Microservices
Test Levels
Rendering
services
for:
 Typical defects and failures of component integration testing:

 Incorrect data, missing data or incorrect data encoding
 Incorrect sequencing or timing if interface calls
 Interface mismatch
 Failures in communication between components
 Unhandled or improperly handled communication between components
 Incorrect assumptions
 Typical defects and failures of system integration testing :

 Incorrect data, missing data or incorrect data encoding
 Incorrect message structure between systems
 Interface mismatch
 Failures in communication between systems
 Unhandled or improperly handled communication between systems

Test Levels
Rendering
services
for:

 Component integration and system integration tests should concentrate on the integration itself, not the functionality
of the individual modules, components or systems
 Component integration testing is often the responsibility of developers
 System integration testing is generally the responsibility of testers. Testers should understand the system architecture
and should influence integration planning
 Integration strategies
 Big-bang strategy – All units are linked at once resulting in a complete system
 Incremental strategy – Based on system architecture (top-down of bottom-up), functional tasks, transaction sequences
or other aspects as well
 In order to ease fault isolation and defects early, integration should normally be incremental rather than “big bang”

Test Levels
Rendering
services
for:
 Top-down integration
- The system is built in stages, starting with components which call other components
- Components which call others are usually placed above those that are called
- Stubs are used for components not yet integrated.
Stub is a passive component,

called by other components.

Test Levels
Rendering
services
for:
 Bottom-down integration
- The opposite of top-down integration.
- Components are integrated in a bottom-up order.
- These are then tested and added to the modules above them to form larger sub-systems which are then tested.
- Bottom-up requires the heavy use of drivers instead of stubs .
Driver is a specially designed user interface which

enables test data to be inputted and passed to a
subsystem which is being tested. Only used in testing
and does not have any place in the final system.

Test Levels
Rendering
services
for:
– System test
• System test focuses on the behavior of the whole system / product in a live environment.
• Independent testers typically carry out the system testing
Functional requirement specifies a function that a system or system component must perform.
Non-functional requirement details how the application will perform in use.
 Installability – installation procedures

 Maintainability – ability to introduce changes to the system
 Performance – expected normal behavior
 Load handling – behavior of system under increasing load
 Stress handling – behavior at the upper limits of system capability
 Portability – use on different operating platforms
 Recovery – recovery procedure on failure
 Reliability – ability of the software its required function over time
 Usability – ease with which users can engage with the system

Test Levels
Rendering
services
for:
– System test
 Objectives
 Reducing risk
 Verifying the behavior of the system
 Validating that the system is complete and will work as expected
 Building confidence in the quality of the system as a whole
 Finding defects in the system
 Preventing defects in higher test levels or production

Test Levels
Rendering
services
for:
– System test
 Test basis:
 System and software requirement specifications

 Risk analysis reports
 Use cases
 Epics and user stories
 Models of system behavior
 State diagrams
 System and user manuals

Test Levels
Rendering
services
for:
– System test
 Test objects:
 Applications
 Hardware/software systems
 Operating systems
 System under test(SUT)
 System configuration and configuration data

Test Levels
Rendering
services
for:
– System test
 Typical defects and failures:
 Incorrect calculations
 Incorrect or unexpected system behavior
 Incorrect control and/or data flows
 Failure to carry out end to end functional tasks
 Failure to work properly in production environment
 Failure of the system to work as described in the system and user manuals

Test Levels
Rendering
services
for:
– Acceptance test
 User Acceptance Test is executed by the customer to verify that the system meets their business needs.
 Operational Acceptance Test: involves checking that the processes and procedures are in place to allow the
system to be used and maintained
- Back-up facilities
- Procedures for disaster recovery
- Training for end users
- Maintenance procedures
- Data load and migration tasks
- Security procedures
 Contract and Regulation Acceptance Test
- Contract acceptance test
- Regulation acceptance test
 Alpha and Beta tests
- Alpha testing takes place at the developer’s site (but not by the developing team)
- Beta testing takes place at the customer’s site in real world working conditions (field test)

Test Levels
Rendering
services
for:
– Acceptance test
 Objectives
 Verifying that the behavior of the system is as specified

 Validating that the system is complete and will work as expected
 Establishing confidence in the quality of the system as a whole
 Defects may be found but this is NOT an objective!

Test Levels
Rendering
services
for:
– Acceptance test
 Test basis:
 Business processes
 User or business requirements
 Regulations, legal contracts and standards
 Use cases
 System requirements
 System or user documentation
 Installation procedures
 Risk analysis reports

Test Levels
Rendering
services
for:
– Acceptance test
 Test basis for operational acceptance testing:
 Backup and restore procedures

 Disaster recovery procedures
 Non-functional requirements
 Operations documentation
 Deployment and installation instructions
 Performance targets
 Database packages
 Security standards or regulations

Test Levels
Rendering
services
for:
– Acceptance test
 Test objects:
 System under test

 System configuration and configuration data
 Business processes for a fully integrated system
 Recovery systems and hot sites
 Operational and maintenance processes
 Forms
 Reports
 Existing and converted production data

Test Levels
Rendering
services
for:
– Acceptance test
 Typical defects and failures:
 System workflows do not meet business or user requirements

 Business rules are not implemented correctly
 System does not satisfy contractual or regulatory requirements
 Non-functional failures

Test Levels
Rendering
services
for:
– Acceptance test
 Acceptance testing is often the responsibility of the customers, business users, product owners or operator of a system
 Acceptance testing is often the last test level, but not always, for example:
 Acceptance testing of COTS software product may occur when it is installed or integrated
 Acceptance testing of a new functional enhancement may occur before system testing
 In iterative development, project teams can have different forms of acceptance testing during and at the end of each
iteration

Rendering
Agenda services
for:

2. Test Levels
3. Test Types

Test Types
Rendering
services
for:
 In different test levels different types of testing are required to meet the overall test objective. Software Test types
are introduced as a means of clearly defining the objective of a certain level for a program or project
Test type is a group of test activities based on specific test objectives aimed at specific characteristics
of a component or system. A test type may also take place on one or more test levels or test phase.

Test Types
Rendering
services
for:
- Functional testing
 Functional testing considers the specific behavior.

 Functional testing and Black-Box Testing is not the same.
- Black-box testing also includes non-functional testing.
- Experience-based Techniques can also be used for functional test case design.
Functional testing: Testing conducted to evaluate the compliance of a component or system with
functional requirements.
 Functional tests focusing on (based on ISO 9126):
- Suitability
- Interoperability
- Security
- Accuracy
- Compliance

Test Types
Rendering
services
for:
- Non-Functional testing
 Non-functional testing focuses on the quality characteristics or non-functional attributes of the test object.
- We are testing something that we need to measure on a scale of measurements
- It is the testing of “how well” the system works
Non-functional testing: Testing conducted to evaluate the compliance of a component or system with
non-functional requirements..
 Non-functional testing considers the external behavior of the software and in most cases uses black-box test design
techniques to accomplish that
 Non-functional testing should be performed at all test levels and done as early as possible
 Non-functional testing can be assessed by Non-functional coverage

Test Types
Rendering
services
for:
- Non-Functional testing
Non-functional test focusing on (based on ISO 9126)
Reliability Usability Efficiency Maintainability Portability
• Maturity • Understandability • Time behavior • Analyzability • Adaptability

(robustness) • Learnability (performance) • Changeability • Install ability
• Fault-tolerance • Operability • Resource • Stability • Co-existence
• Recoverability and • Attractiveness and utilization and • Testability and • Replaceability and
compliance compliance compliance compliance compliance

Test Types Rendering
services
for:
- White Box testing
 Structural testing based on the internal structure of a system or component

- Also called as white-box testing.
Structural testing or White-box testing based on an analysis of the internal structure of the
component or system.
 In different test levels different kind of structure used to derive test cases
- Lower test levels (component and component integration tests): system architecture
- Higher test levels (system and acceptance tests): business model or menu structure
 Coverage techniques helps us to define exit criteria
 The test design technique used for structural testing are structure-based or white-box techniques

Test Types Rendering
services
for:
- Testing Related to Changes
 When a defect is detected and fixed then the changed software should be retested to confirm that the problem has
been successfully removed
- Re-testing
Re-test demonstrates that the reported defect has been fixed.
 The unchanged software areas should also be retested to ensure that no additional defects have been induced as a
result of changes to the software
- Regression testing
Regression test checks that there are no additional problems in previously tested software.
• Regression testing should also be carried out if the environment has changed
Operation System
• Regression testing involves the creation of a set of tests which serve to
Changed
Changed demonstrate that the system works as expected.
function
function

Test Types and Test Levels Rendering
services
for:
 It is possible to perform any of the test types at any test level
 It is not necessary, for all software, to have every test type across every test level
 It is important to run applicable test types at each level, earliest level where the test type occurs

Rendering
Agenda services
for:

2. Test Levels
3. Test Types

Maintenance Testing Rendering
services
for:
 For many projects the system is eventually released into the live environment
 During the deployment it may become necessary to change the system
 Changes may be due to:
- Additional features being required
- The system being migrated to a new operating platform
- The system being retired
- New faults being found
- Configuration data changed
 Testing which takes place on a system which is in operation in the live environment is called Maintenance Testing.
Operation System
DB/HW change Maintenance test
Add or change
Changed
feature
function

services
for:
- Triggers
 We can classify the triggers for maintenance testing:
 Modification:
 Planned enhancements
 Corrective or emergency changes
 Changes of operational environment
 Upgrades of COTS
 Patches for defects and vulnerabilities
 Migration:
 One platform to another
 Data migration/ conversions
 Retirement – application reaches end of life
 For IOT systems, can be triggered by introduction of completely new or modified things: software or hardware
devices, into the overall system. In this case we have a particular emphasis on integration testing at different levels.

services
for:
- Impact Analysis
 Impact Analysis evaluate the changes that were made for a maintenance release to identify :
 Consequences
 Expected and possible side effects of a change
 Areas in the system that will be affected by the change
 Impact on existing tests
 Impact Analysis can be done before a change is made to help decide if the change should be made

services
for:
- Impact Analysis
 Impact analysis can be difficult if:

 Specifications are out of date or missing
 Test cases are not documented or are out of date
 Bi-directional traceability between tests and test basis has not been maintained
 Tool support is weak or non existent
 The people involved do not have domain and/or system knowledge
 Insufficient attention has been paid to software’s maintainability during development
 Maintainability Testing is different.

- The term maintainability corresponds to the ability to update or modify the system under test.
- This is a very important parameter as the system is subjected to changes throughout the software life cycle.

Check of understanding
Rendering
services
for:

Rendering
services
for:

03 – Static Techniques

Agenda Rendering
services
for:
1. Static Techniques and the Test Process

2. Review Process
2.1. Activities of a Formal Review
2.2 Roles and Responsibilities
2.3 Types of Reviews
2.4 Success Factors for Reviews

Agenda Rendering
services
for:

2. Review Process

Static Techniques and the Test Process Rendering
services
for:
Static Testing = Testing of a software development artifact without execution of these artifacts, e.g., reviews or static
analysis.
Benefits of reviews:
 To identify errors as soon as possible in the development lifecycle

 Finding omissions and errors in the software specifications
 Reduce testing cost and time
Objectives of the reviews:
Both static analysis and dynamic testing have the same objective – identifying defects.
They are complementary: static techniques find causes of failure rather than the failure itself.

services
for:
What to
review?

services
for:

services
for:
Require
ments Design
Web
specific
pages
ations
User Any
Code
guides software
work
product
Test Test
scripts plans
Test
Test
specific
cases
ations

services
for:
Typical defects found with static testing:

 Deviations from standards
 Requirement defects
 Design defects
 Insufficient maintainability
 Incorrect interface specifications
 Coding defects
 Security vulnerabilities
 Gaps and inaccuracies in the test basis traceability or coverage
 Most maintainability defects can be found only by static testing.

Agenda Rendering
services
for:

2. Review Process

Review Process Rendering
services
for:
The types of review vary from informal to systematic .
The formality of a review process depends on factors such as:
 Software development life cycle model

 Maturity of the development process
 Any legal or regulatory requirements
 The need for an audit trail
 Complexity of the work products

Review Process Rendering
services
for:
The types of review vary from informal to systematic .
Objectives:
 Finding defects
 Gaining understanding
 Generating discussions
 Educating participants such as testers and new team members
 Making decision by consensus

Agenda Rendering
services
for:

2. Review Process

Activities of a Formal Review Rendering
services
for:
Issue
Individual Fixing and
Planning Initiate review communication
review reporting
and analysis
 Defining the scope (purpose, documents, parts of documents to review, quality)

 Estimating effort and timeframe
 Identifying characteristics (review type, roles, activities and checklists)
 Selecting the people and assigning roles
 Defining the entry and exit criteria ( for more formal review types)
 Checking entry criteria (for more formal review types)

services
for:
Issue
Planning Initiate review review
communication
reporting
and analysis
 Distribute the work products

 Explain scope, objectives, process, roles and work
products to the participants
 Answering any questions that participants may have

services
for:
Issue
Planning Initiate review
review
communication
reporting
and analysis
 Reviewing all or part of the work product

 Noting potential defects, recommendations and questions

services
for:
Issue
Planning Initiate review
review
communicatio reporting
n and analysis
 Communicating identified potential defects

 Analyzing potential defects, assigning ownership and status to them
 Evaluating and documenting quality characteristics

 Evaluating the review findings against exit criteria to make a review decision :
• Reject
• Major changes needed

• Accept
• Possibly with minor changes

services
for:
Issue
Planning Initiate review communication
review
and analysis reporting
 Creating defect reports for those findings that require changes

 Fixing defects found (by author)
 Communicating defects to the appropriate person or team

 Recording updated status of defects
 Gathering metrics
 Checking on exit criteria
 Accepting the work product when the exit criteria are reached

Agenda Rendering
services
for:

2. Review Process

Roles and Responsibilities Rendering
services
for:
The roles during a review are:
• Decides on the execution of review

 Manager
• Is responsible for review planning
 Moderator/Facilitator • Assigns staff, budget and time

• Monitors ongoing cost- effectiveness
 Review leader
• Executes control decisions
 Author
 Reviewer/Checker/Inspector
 Scribe/Recorder :

services
for:
• Ensures effective running of the

 Manager
review meeting (when held)
 Moderator/ Facilitator • Mediates, if necessary, between

the various points of view
 Review leader • Success of the review depends on
him
 Author

services
for:
• Takes overall responsibility of the

 Manager
review
 Moderator/ Facilitator • Decides who will be involved

• Organizes when and where it will
 Review leader take place
 Author

services
for:
• Creates the work product

 Manager
under review
 Moderator • Fixes defects in the work

product under review (if
 Review leader
necessary)
 Author

services
for:
 Manager • Individuals with specific

technical or business
 Moderator
background
 Review leader • Identify and Describe potential
defects/findings
 Author
• Participates in review meeting
 Reviewer/Checker/Inspector • May represent different

perspectives

services
for:
 Manager • Collates potential defects found

during individual review activity
 Moderator
• Records new potential defects,
 Review leader open points and decisions from

the meeting
 Author
 Scribe/Recorder

Agenda Rendering
services
for:

2. Review Process

Review Process – Types Rendering
services
for:
Ins
pec
tion
ty
ali
m
Technical review
for
of
el
v
Le
Walkthrough
Informal Review

Review Process – Informal Review Rendering
services
for:
• No formal process
• May take the form of pair programming or technical lead review
Description • Varies in usefulness depending on the reviewers
• The results may be documented, but it is not required
• Very commonly used in Agile
• Author
Roles
• Reviewer
• Generating new ideas

Main purpose • Quickly solving minor problems
• Find defects

Review Process – Walkthrough Rendering
services
for:
• A step-by-step presentation by the author of a document in order to gather

information and to establish a common understanding of its content
• Led by author, Scribe is mandatory
• May take the form of scenarios, dry-runs and peer group participation
Description • Use of checklists is optional
• May vary in practice from quite informal to very formal
• The results may be documented, but it is not required
• Open – ended sessions with optional pre meeting preparation
• Author
Roles • Multiple reviewers
• Scribe (Mandatory)
• Find defects
• Improve software product,
Main purpose • Consider alternative implementations
• Evaluate conformance to standards
• Gaining understanding, Learning, achieving consensus

Review Process – Technical Review Rendering
services
for:
• Formal Defect detection process

• May take the form of pair peer review without management participation
• Pre-meeting preparation by reviewers
Description • Optional use of checklists
• The results are documented in review report, which includes list of findings,
recommendations related to findings and an verdict of compliance
• May vary in practice from quite informal to very formal
• Author
• Trained moderator (not the author)
Roles • Scribe
• Multiple reviewers (technical peers or technical experts)
• Find potential defects, gaining consensus

• Evaluating quality and building confidence in the work product
• Preventing future similar defects through author learning and root cause analysis
Main purpose • Motivating and enabling authors to improve work products and software
development process

Review Process – Inspection Rendering
services
for:
• Formal process based on checklists, specified entry and exit criteria

• Usually takes the form of peer examination
• Dedicated, precise roles
Description • Pre-meeting preparation by reviewers
• Includes metrics gathering
• The results are documented in inspection report, which includes list of findings,
recommendations related to findings and a verdict of compliance
• Formal follow up process
• Author
• Trained moderator (not the author)
Roles • Scribe
• Multiple reviewers
• Reader (optional)
• Find potential defects

• Evaluating quality and building confidence in the work product
Main purpose • Preventing future similar defects through author learning and root cause analysis
• Motivating and enabling authors to improve work products and software
development process

Review Techniques Rendering
services
for:
Examples of individual review techniques:
 Ad hoc: A review technique carried out by independent reviewers informally, without a structured process.
 Checklist-based : A review technique guided by a list of questions or required attributes.
 Scenarios and dry runs: A review technique where the review is guided by determining the ability of the work
product to address specific scenarios.
 Role-based: A review technique where reviewers evaluate a work product from the perspective of different
stakeholder roles.
 Perspective-based: A review technique whereby reviewers evaluate the work product from different viewpoints.

Agenda Rendering
services
for:

2. Review Process

Success Factors for Reviews Rendering
services
for:
 Each review has a clear predefined objective.
 Review types are suitable to achieve the objectives
 Review techniques are applied that are suitable to the type and level of software work products and reviewers.
 Checklists used address the main risks and are up to date
 Large documents are written and reviewed in small chunks
 Participants have adequate time to prepare
 Reviews are scheduled with adequate notice
 Management supports a good review process (e.g. by incorporating adequate time for review activities in project
schedules).

People Related Success Factors for Reviews Rendering
services
for:
 The right people for the review objectives are involved
 Defects found are welcomed, and expressed objectively
 Testers are seen as valuable reviewers
 Participants dedicate adequate time and attention to details
 Reviews are conducted on small chunks, not to lose focus
 The meeting s well managed
 The review is conducted in an atmosphere of trust
 Participants avoid body language and behavior that might indicate boredom, exasperation or hostility
 Training is given in review techniques, especially the more formal techniques, such as Inspection
 There is an emphasis on learning and process improvement

Agenda Rendering
services
for:

2. Review Process

8. Check of Understanding Rendering
services
for:

04 –
Test
Desig
n
Tech Rendering
nique services
s
for:

– Test Design Testing

Agenda Rendering
services
for:
1. Test Conditions and Designing Test Cases

2. Categories of Test Case Design Techniques
3. Specification-Based or Black-Box Techniques
4. Structure –Based or White-Box Techniques
5. Experience-Based Techniques
6. Choosing Test Techniques
7. Summary
8. Check of Understanding

Agenda Rendering
services
for:

7. Summary

Rendering
1. Test Conditions and Designing Test Cases services
for:
IEEE 829 – Standard for software and system test documentation
Published in 1998-New version 2008

Specifies the form of a set of documents for use in eight defined stages of software testing
Documents included:
• Test Plan
• Test Design Specification
• Test Case Specification
• Test Procedure Specification
• Test Item Transmittal Report
• Test Log
• Test Incident Report
• Test Summary Report
The standard specifies the format of these documents

No specification that all of them must be produced
No specification of any criteria regarding adequate content for these documents
Test Conditions and Designing Test Cases Rendering
services
for:
Related Standards
IEEE : Institute of Electrical and Electronics Engineers

ISO : International Organization for Standardization
BS: British Standards
IEEE 1044 : -standard for the classification of software anomalies

ISO 9126 : -covers software engineering aspects of product quality
--------------------------------------------------------------------------------------------------------------------------------
IEEE 1012 : Standard for Software Verification and Validation
IEEE 1028 : Standard for Software Inspections
IEEE 12207: Standard for software life cycle processes and life cycle data
BS 7925-1 : vocabulary of terms used in software testing
BS 7925-2 : standard for software component testing

Rendering
E
Test Conditions and Designing Test Cases services
for:
Test Design Specification
 Test Design steps :

• Identify Test Item What I would like to test ?
• Identify Test Condition What condition do I want to check ?
• Specify Test Case How to be tested?
o Set of Input Values
o Execution Preconditions
Test Object EHCU
o Expected Results Test Item
ABS
o Execution Post conditions Test Scenario
Verify that ABS gets active in case an emergency
Test
brake case
is detected.
• Specify Test Procedure ABS maneuver with brake 100 bar during second
20-30 Abs shall get active when one of the following
Test condition
conditions is met.:-wheel is in excessive slip,etc…
If wheel slip is
Environment detected then ABS
gets active.
Test execution
E
services
Test Documents – IEEE 829

for:
Organization Level
Testing philosophy of
Organization Level
company
Standard
• Test Policy
Project Level
Test
Policy How we test / why
• Test Strategy we TE
test?
• Based on Test Design • A testSpec.
log is aTDE
chronological
• Test Plan • Test
• Execution objective
The Test Evaluation Report PM
• Requirements Test Phase
traceability Project
from Level
Test
record TE
of relevant details
TM in cooperationcollects,
with • TCO •
Deliverables Features
organizes, and presents to be tested Level
TDEBasis about the execution of tests
• Test Evaluation Report Test Design Test Case Test Log derived from Test
Test • Risk and • Test conditions
mitigation
the Test Results and key measures •
• Contains time samples of
Describes test casesmultiple In detail
signalswith
acquired
Specification
• Testing Specification Basis for each features
Strategy measurements
of test and metrics
to enable objective quality
necessary test steps and acceptance
during measurement
• Training
evaluation and plan
assessment • Data acquisition systems :
criteria’s
• Way Vector
Test Phase Level of reporting,
tools
TE responsibilities:
Test
• items
Reviewing TM Test trackingTestand
(CANalyzer,CANape,CANoe)
• Test Design Specification • Summarizes the test results the Test Results,
of all Summary
TDE TE
Evaluation
• Features
change request to statistics,
be testedofand
• aSequence
managing
test centers in order to give •actions for the
Describes execution
theReport
items being delivered for testing
Report
• Test Case Specification • Test techniques incidents
Test rating for thecoverage
Test release statistics.
Test Item Test Incident
of a test • Where to find them
•
Procedure •
Reviewing Responsibilities
•important
DetailedChange
Transmittal •
Report for setting up the test
method What is new about occurring
them at a
• Test Procedure Specification Plan recommendation. • • Gives approval for theircertain release
RequestReport
Specification Entry Criteria
and Issue details. , running test sets and
environment testing
• Generated at the • end of• the testaccurate • Provide testers warranty that the items are fit to
• Test Item transmittal Report Presenting ExitanCriteria and fair
evaluating results.
be tested and give clear mandate to start level testing
execution assessment
phase• Schedule of the software based • Summary
• Test Log on the defined Evaluation Mission. description of the
• Test Incident Report actual incident
• Test Summary Report
Preparation of tests Running the tests Completion of testing

services
for:
Test Documents – Test Design Specification
 Test Design Specification –Document specifying the test conditions (coverage items) for a test item, the detailed test
approach and the associated high level test cases.
 Test Design Specification Template

Test Identification
• Test design specification number Approach
Test design refinements
specification number
• Identification of each test case with a short
•Features
Selection
Namingto of
bespecific
follow same test
tested techniques
general rules as the software
• description of the case, it’s test level or any other
Features to be tested Feature
• Group • Pas/Fail
Reason
it is related
related Criteria
for
to test technique selection
items together
appropriate information required to describe the
• Describe
Method(s)
Version date
Reference thetocriteria
for result
and for assessing
analysis
the original
version (Tools)the feature
documentation
number whereor
• Approach refinements test relationship
• set of features
(Eg.automated and
test whether theobtained)
scripts)was test(s) were
• this
Version
test author
objectiveand (feature
contact information
Identification of each test procedure with a short
• Test identification • successful
Relationship
Revision historyorof
not.
the test item/features to the levels
description of the case it’s level or any other
E.g. this test is passed if ….
of testing
• Feature pass / fail criteria appropriate information required to describe the
test relationship

services
for:
Test Case Specification
 Test Case Specification –Document specifying a set of test cases (objective, inputs, test actions, expected
results and execution pre- and post condition).
 Test Case Specification Template

Test case specification identifier
• Test case specification identifier Input
• Naming specifications
follow same general rules as the software
• Test items Inter-case
•Environmental
TestIdentify
it isitems
relatedDependencies
all inputs required to execute the test
to needs
•Output
Identify
case.
Special
Hardware
Version any prerequisite
specifications
procedure
items
date and versiontest
requirements
or features to cases.
number
be tested by this test
• Input specifications • It
Canis also
Identify
Version
case recommended
•beConfigurations
simplified
all
any outputs
author andusing
special thatinformation
required
tablesthe
to
constraints
contact relationship
verify
on test of
thetest
the case
test
• Can
case(s).
Revisioncases to be
•beLimitations
simplified documented
history using tables at both ends of the
• Output specifications
• relationship.
Software
• Environmental needs
• Special procedure requirements
• Inter-case dependencies

Rendering
Test Conditions and Designing Test Cases services
for:
Test Documents – Test Procedure Specification
 Test Procedure Specification –Document specifying a sequence of actions for the execution of a test.
 Test Procedure Specification Template

Test procedure specification identifier
• Test procedure specification identifier Special Requirements
• Naming follow same general rules as the software
• Manual or Automated
• Purpose Purpose
it is related to
• Test Environment
Steps
•• Describe
Version date the and
purpose
version
of the
number
procedure
• Special requirements •• What
Specialareskills or Training
the activities required with the
associated
•• Version
Refer toauthor
the testand
cases
contact
being
information
executed
• procedure.
Any prerequisite procedures
Include any of the following activities
• Procedure steps • Revision history
that may be required to execute the procedure

E
services
for:
Test Documents – Overview
Test Plan Test Design Test Case Test Test Incident Report Test Summary
Specification Specification Procedure Report
Specification
Unique identifier Unique identifier Unique identifier Unique identifier Unique identifier Unique identifier
Suspension criteria and Features to be tested Test items Purpose Summary Summary
resumption requirements • Version level of test items,
Reference to other relevant
documents
Test Approach Refinements Input specifications Steps for executing Incident Description: Variances
deliverables/Features to a set of test cases Inputs
be tested Expected results
Actual results
Testing tasks Anomalies
Date and time …..
Test Strategy
Responsibilities Test identification Output specifications Impact Comprehensive

assessment
Environmental needs Feature pass-fail Environmental needs Description Summary of results

criteria
• Staffing and training Special procedure Evaluation

needs requirements
• Schedule
Risks and contingencies Inter-case dependencies Summary of activities

services
for:
Test Development Process
 Fields of the test development process

• Test Design
• Test Case Specification
Traceability
Test Execution Schedule
• Test Procedure Specification • Function
Scheduleconnected
test execution
to requirement
(Run at different times)
• Test Execution Schedule
• Traceability

services
for:
Traceability
Capture Software
and development
Requirements
Main follow environment
concern traces
• Requirements traceability
Vertical traceability
• Documenting the life of a requirement • Identifies the comeorigin of items ( e.g. customer
Requirements
Horizontal traceability from different sources
• Find the origin of each requirement •• needs)
Business and follows
person
Identifies thethe these
ordering
relationship same
the items
product
among related as they
(Customer)
items
Documenting life of a requirement
Prioritize
Documenting
• travel trough
Marketing theManager
requirement
the
the relationships
process of software
• across workto groups
find theororigin
product components for
• Requirements come from different sources • Possibility
Determining
Between
development.
Actual requirements
user how valuable
(Driver) and
of each
the
other
requirement
requirement
developmentis to a
the purpose
and track of avoiding
every change potential
which was conflicts.
made to this
• Prioritize the requirement • Eg.
specific user
artifacts
requirement
• Req.
We need
• E.g.: to findSpecification
Associated therequirements
closest requirement from the
across work
• Documenting the relationships user • Software
(ex: Business Analyst)
groups avoiding conflicts
• Set documents test process
• Vertical and horizontal traceability
• Check requirements in Test Case
Specification

services
for:
The idea of Test Coverage
Test Coverage
 Test Coverage
• Degree, expressed as percentage,
 How effective the testing to which a specified coverage item
Coverage measures
has been exercised by a test suite
• Quantitative measure •How
Part of completion
effective criteria defined in
the testing
(Collection of test cases that are
• the
WhatTest
hasPlan (first
been step of the
achieved
• Estimation intended to be used to test a
Fundamental
(Percentage). Test Process).
software).
 Test coverage can be applied to any systematic technique • Used to determine
Estimation how much whenmoreto stop
testing
testing
needs to in be
thedone.
final step of the
 Coverage measures • Eg.: Executed Testcases vs. full nr.
Fundamental Test Process.
of test cases.
• Executed Requirements vs. full nr.
of requirements.

Agenda Rendering
services
for:

7. Summary

2. Categories of Test Case Design Techniques Rendering
services
for:
Test Case Design Techniques
Specification-based or Structure-based or Experience-based

Black-box techniques White-Box techniques techniques
5 2 2
• Equivalence partitioning
• Statement testing • Error guessing
• Boundary value analysis
• Decision testing • Exploratory testing
• Decision table testing
• State transition testing
• Use Case testing

(Scenario testing)

Categories of Test Case Design Techniques Rendering
services
for:
Definition of Different Test Case Design Techniques
Specification-based or Black-box techniques

• Deriving Test Cases based on - an analysis of the specification of a system or component
- some kind of model of what the system should do
• Coverage: based on items tested in the test basis and technique applied to the test basis
Structure-based or White-box techniques

• Deriving Test Cases based on analysis of the internal structure of a system or component
• Information about how the software is constructed is used to derive the Test Cases (code and design)
• Coverage: based on the items tested on a selected structure
Experience-based techniques
• Deriving Test Cases from the tester’s experience (or developers, users or other stakeholders) of similar
systems and general experience of testing.
• Knowledge about likely defects and their distribution

Agenda Rendering
services
for:

7. Summary

3. Specification based or Black-box techniques Rendering
services
Introduction for:
 Derive Test Cases

• From the specification, requirements, Specification Design
use cases or user stories
• From model
• What the system should do
 Black-box techniques
• Use Case testing (Scenario testing)

Specification based or Black-box techniques Rendering
services
for:
Equivalence partitioning – Input partitions
 Reduces the number of needed Test Cases

 ‘chunked’ into groups of similar inputs
 Example : if a program accepts any integer values between -10.000 and +10.000 (inclusive)
• Valid Equivalence Partitions ? [-10.000- 10.000]
• Non –valid Equivalence Partitions ?
<-10.000
>10.000

services
for:
Equivalence partitioning – Output Partitions
 Output can be partitioned as well

 Example : We have a bank account program which offers variable interest rates
• 0.5% for the first $1.000 credit;
• 1% for the next $1.000 credit;
• 1.5% for the rest
 We can use Test Cases that generate each

of these outputs as alternatives to generate input partitions
• Valid Equivalence Partitions? >$0- $1000
• Non-valid Equivalence Partitions?
$1000.01- $2000
$2000.01- $
<0$
0$

Rendering
Specification based or Black-box techniques services

for:
Black-Box Techniques – Boundary Value Analysis
 Mistakes made by programmers cause errors that tend to cluster around boundaries
 The maximum and minimum values of a partition are its boundary values
• Valid boundary value
• Invalid boundary value
 Example: The partition is [-100; +100] (inclusive) - integer

• Valid boundary values?
• Invalid boundary values?
-100
100
-101
101

services
for:
Black-Box Techniques – Decision Table Testing
• Good to use for system requirements that contain logical conditions (business rules)
• Decision table contains all the
• Input conditions
• Actions that may arise from them
Rules involve combinations of conditions to produce some combinations of actions
Business rule 1 Business rule 2 Business rule 3
Condition 1 TRUE TRUE FALSE
Condition 2 TRUE - TRUE
Condition 3 TRUE FALSE FALSE
Action 1 YES YES YES
Action 2 NO NO YES

services
for:
Exercise– Decision Table Testing
Loyalty card holders enjoy

Rule 1 Rule 2 Rule 3 Rule 4 the benefits of either
additional discounts on all
Customerwith
Customer with loyalty
loyalty Card False False True True purchases (rule 3) or the
Card acquisition of loyalty points
Extra (rule 4), which can be
Extradiscount
discountselected
selected - - True False
converted into vouchers
Spent for the supermarket or to
Spent >>100$
100$ False True - -
equivalent points in
Actions
Actions: : schemes run by partners.
Customers without a loyalty
Extra
Extradiscounts
discounts False True True False card receive an additional
discount only if they spend
Loyalty
Loyaltypoints
points False False False True more than $100
on any one visit to the store
(rule 2) , otherwise only the
special offers offered to all
customers apply (rule 1).

services
for:
Black-box techniques – State Transition Testing
 Useful technique when actions are triggered by changes of the input conditions, or changes of ‘state’
 The basis of the test is the system’s State Transition Diagram
 Testers can analyze the behavior in terms of what happens when a transition occurs
• Transition caused by events
 All possible events and states recorded in a state table
• Shows the relationships between states and inputs
 What tests can be designed to
• Cover typical sequence of states
• Cover every state
• Test invalid transitions
 When useful ?
• Testing embedded software (E.g. MKB test)

E
services
State Transition Diagram Example for:
Mode Set
Change display
to Altimeter
Mode = Mode =
Time Altimeter
Mode
Change display
Change display
to Time
to Set Hrs
Set
Mode
Change display
to Time
Mode =
Mode =
Set
Set Hrs
Mode Mins
Change display
to Set Mins Set
Set Add 1 to Mins
Add 1 to Hrs
services
State Transition Table Example for:
Mode Set
Mode = Time Mode = Altimeter Mode = Set Hours
Change Display to Altimeter Change Display to Set Hours
Mode = Altimeter Mode = Time N

Change Display to Time
Set Hours Mode = Set Mins Set Hours

Change Display to Mins Add 1 to Hours
Set Mins Mode = Time

Set Hours
Change Display to Time
Add 1 to Mins.

services
Use Case Testing for:
 Use Cases are descriptions of interactions between users (actors) and the system in a high level view of the
requirements
 The main advantage is that we exercise real user processes or business scenarios
 The main principles applied elsewhere can be applied, also here
• Take typical examples

• Exercise some attempts at incorrect process flows
• Exercise the boundaries
 Use cases may be described at the abstract level or at the system level
 Each Use Case has preconditions and post-conditions
 Useful for designing acceptance tests with customer participation

E
services
for:
Use Case Testing Example
User
Basic Flow
Alternate Flow 1
Alternate Flow 2
Alternate Flow 4 Alternate Flow 3
End Use Case 2 End Use Case 3
End Use Case 1

Agenda Rendering
services
for:

7. Summary

Structured based or White-box techniques Rendering
services
for:
White-Box Techniques - Background

 Based on an identified structure of the software or system
 Test basis: code, software architecture, detailed design or any other information regarding the structure of the code
 Coverage varies on different levels of structure
Code Design
 Structural testing techniques for code coverage

• Decision Testing
• Statement Testing
Code Flow Chart Control Flow Graph

services
for:
White-Box Techniques - Code Example
Program MaxSelector
A, B, C, Max: Integer
Begin
Read (A);
Read (B);
A B C
Read (C);
If (A > B)
Then
If (A > C)
Then
Max = A;
Else
Max = C;
Endif
Else MAX
If (B > C)
Then
Max = B;
Else
Max= C;
Endif
Endif
Print (“The Maximum of A, B and C is : “, Max);
End
services
for:
Begin White-Box Techniques - Flow Chart Example
Read(A)
Read(B)
Read(C)
N
A>B B>C
Y
N Y N
A>C
Y
Max = A Max = C Max = B Max = C
Print(“The Maximum of A, B and C is: *, Max)
174
End BeeSpeed Timisoara Engineering Centre, 2017
services
for:
White-Box Techniques - Control Flow Graph Example
Y N
If A > B
If A > C If B > C
Y N Y N
Endif Endif
Endif

Rendering
Structured based or White-box techniques services
for:
White-Box Techniques – Statement Testing and Coverage

• Aim : Exercise programming statements, only executable statements counts
X 100%
• Achieving 100 % statement coverage doesn’t mean we have tested everything

• Statement coverage just provides a baseline, it is too weak to be considered as an adequate measure of test
effectiveness

Rendering
E
Structured based or White-box techniques services
for:
White-Box Techniques – Statement Testing and Coverage (Example)

How many test cases are needed to achieve 100% statement coverage?
If ( ( temperature < 0 ) or (temperature > 100 Advantages Disadvantages

)) { alert (“ DANGER “ ) ; if ( ( speed
> 100 ) and ( load <= 50 ) ) { speed = 50; • It verifies what the written code • It cannot test the false
} } else { is expected to do and not to do conditions.
check = false; } • It measures the quality of code • It does not report that
written whether the loop reaches its
termination condition.
• It checks the flow of different • It does not understand the
paths in the program and it logical operators.
also ensure that whether those
A. 5 path are tested or not
B. 4
C. 2
D. 3

services
for:
White-Box Techniques – Decision/Branch Testing and Coverage
• Aim : Exercise programming decisions, both true and false exits of the condition (also possible decision outcomes)
X 100%
• Complete decision test also doesn’t guarantee revealing of all mistakes in the code

E
services
for:
White-Box Techniques – Decision/Branch Testing and Coverage
(Example)
• How many test cases are needed to achieve 100% decision coverage?
If ( p = q) { s = s + 1;
if ( s < 5 ) { t= Advantages Disadvantages
10; } } else if ( p > q ) { • To validate that all the • This metric ignores branches
t = 5; } branches in the code are within Boolean expressions
reached which occur due to short-
circuit operators.(e.g break)
• To ensure that no branches
lead to any abnormality of the
program’s operation
A. 3 • It eliminate problems that occur
with statement coverage
B. 6 testing
C. 5 • 100% decision coverage
guarantees 100% statement
D. 4 coverage, but not the other
way around.

services
for:
White-Box Techniques – (Modified) Condition/Decision Coverage

 A condition is a leaf-level Boolean expression
 Condition coverage (or Predicate coverage) (All conditions and All outcomes)
 Multiple condition Coverage (Has every possible combination of Boolean sub-expressions executed ?)
Example : (a==b) && (c > d)
 MCDC (Modified Condition/Decision Coverage)

• Each condition in a decision is shown to independently affect the outcome of a decision
• Every basic condition is independent of others ( true/false by switching only one condition)]
• Independence of a condition is shown by proving that only one condition changes at a time (E.g. Test Design
Specifications).

services
for:
White-Box Techniques
In order to ensure – (Modified)
Condition coverage Condition/Decision
criteria for this example, A, B and C Coverage (Example)
should be evaluated at least one time "true" and one time "false" during tests,
which would be the case with the 2 following tests:
1. A = true / B = true / C = true
2. A = false / B = false / C = false
In order to ensure Decision coverage criteria, the condition ( (A or B) and C )
should also be evaluated at least one time to "true" and one time to "false".
Indeed, in our previous test cases:
3. A = true / B = true / C = true ---> the decision is evaluated as "true"
4. A = false / B = false / C = false ---> the decision is evaluated as "false“
However, these two tests do not ensure a Modified condition/decision coverage which implies that each boolean variable should be
evaluated one time to "true" and one time to "false", and this with affecting the decision's outcome. (if 3 Boolean conditions we need 4 test
cases)
5. A = false / B = false / C = true ---> decision is evaluated to "false"
6. A = false / B = true / C = true ---> decision is evaluated to "true"
7. A = false / B = true / C = false ---> decision is evaluated to "false"
8.
181 A = true / B = false / C = true ---> decision is evaluated to "true" BeeSpeed Timisoara Engineering Centre, 2017
E
services
for:
White-Box Techniques – Other Structure-Based Techniques
 For different systems different coverage measures are essential

• Entry/exit Coverage (Has every possible call and return of the function been executed?)
• Function Coverage (Has each function in the program been called ?)
• Path Coverage (Has every possible route through a given part of a code been executed? )
• Linear Code Sequence and Jump ( LCSAJ) Coverage
o Variation of path coverage, it executes only sub-paths that can easily be represented in the
program source code
• Data Flow Coverage
o Focusing on the data flow- data goes from one subsystem to another –
o check Interfaces
• Object Code Branch Coverage
• Loop Coverage etc.

Agenda Rendering
services
for:

7. Summary

E
5. Experience-Based Techniques Rendering
services
for:
 Useful when testes have not enough time to execute a full structured test set
Experience Design
 Experience-based testing techniques

• Error guessing ( Guess each error that can appear)
• Exploratory testing
• Checklist-based testing

Agenda Rendering
services
for:

7. Summary

6. Choosing Test Techniques Rendering
services
for:
 We must use different test techniques
 Which test techniques ? depends on a number of factors, including internal and external factors
 Testers generally use a combination of test techniques
• (Process , Rule , Data-driven techniques to ensure adequate coverage of the object under test)

Choosing Test Techniques Rendering
services
for:
Internal and external factors
Internal factors ( Product related) External factors (Project related)
Models used Level and type of risk
Tester knowledge / experience Customer / contractual requirements
Likely defects Type of component or system
Test objective Regulatory requirements/standards
Documentation Time and Budget
Life cycle model
Previous experience of types of defects found

Agenda Rendering
services
for:

7. Summary

Categories and types of Test Case Design Techniques Rendering
E
services
for:
Test Case Design Techniques
Specification-based or Structure-based or Experience-based

Black-box techniques White-Box techniques techniques
5 2 3
• Statement testing • Error guessing
• Decision testing • Exploratory testing
• Checklist Testing
• Use Case testing

(Scenario testing)

Agenda Rendering
services
for:

7. Summary

8. Check of Understanding Rendering
services
for:

Rendering
services
for:

– Test Management

Agenda Rendering
services
for:
1. Test Organization
2. Test Planning and Estimation
3. Test Progress Monitoring and Control
4. Configuration Management
5. Risk and Testing
6. Defect Management

Agenda Rendering
services
for:
5. Risk and Testing

Test Organization Rendering
services
for:
Test
Structure Roles
Organization

Test Organization – Level of independence Rendering
services
for:
Independent testers external to the

organization ( in/outsourcing)
Independent testers from the business

organization or user community
Independent team from within the

organization
Tester and developer in the same team
Tests done by the developer

Test Organization – Independence Rendering
services
for:
PRO CONS Risk of isolation

from the
Testers are not development
influenced by team
the other
project
members Communication
issues
Developers can
Can act as ‘the loose the
customer’s ‘quality
voice’ ownership’
attribute
More objectivity
in evaluating
the product
quality issues

Test Organization – Testing Roles Rendering
services
for:
In this syllabus there are two test positions covered:
 Test leader (test manager or test coordinator)

 Tester
Testing tasks may be done by people in specific testing role, or can be performed by someone in another role (e.g. Project
manager, quality assurance manager, development manager, etc)

Test Organization – Tasks of a Test Leader Rendering
services
for:
Responsibilities Plan, estimates test effort, time and cost, collaborates with project manager
of Test Leaders
include: Writes and reviews the test strategy and test policy
Write and update the test plans
Initiate test analysis, design, implementation, execution
Set-up configuration management of test environment & deliverables
Monitors and controls the execution of tests, checks the status of exit criteria
Chooses suitable test metrics
Decides if and to what degree to automate the tests
Select tools and organize trainings
Schedule tests
Prepare summary and progress test reports
Adapt planning based on test results and progress

Test Organization – Tasks of a Test Leader Rendering
services
for:
Responsibilities Create training plans for testers

of Test Leaders
include:
Promote and advocate the testers within the organization
Develop the skills and careers of testers
Share testing perspective with other project activities

Test Organization – Tasks of a Tester Rendering
services
for:
Responsibilities Review and contribute to test plans

of a Tester
include: Analyze, review and assess user requirements, user stories, acceptance criteria,
specifications and models for testability
Identify and document test conditions and capture traceability between test cases, test
conditions and the test basis
Design, set up and verify test environment
Prepare and acquire test data
Design, implement, execute and log the tests, evaluate the results and document problems
found
Use required tools
Create the detailed test execution schedule
Automate tests
Measure performance of components and systems
Review tests developed by others

Agenda Rendering
services
for:
5. Risk and Testing

Test Planning Rendering
services
for:
Test planning is the first stage in test process, it defines the scope of testing and the test
completion criteria which determine when we stop to test. It covers development,
implementation projects and maintenance activities and is influenced by project
organization. (risk analysis performed)
Test planning is a continual activity that spans the life of the test project; it takes place in all
life-cycle stages.

Test Planning Activities Rendering
services
for:
 Determine the scope and the risks that need to be testes.

 As well identifying and agreeing the objectives of the testing.
 Putting together the overall approach of testing
 Liaison with the project manager and making sure that the testing activities have been included within the software life-
cycle activities such as:
a) Design - The development of the software design;

b) Development - The building of the code;
c) Implementation - The activities surrounding implementation into a live environment.

Test Planning Activities Rendering
services
for:
 Decide what needs to be tested, what roles are involved and who will perform the test activities, planning when and
how the test activities should be done, deciding how the test results will be evaluated, and defining when to stop testing
(exit criteria)
 Scheduling test analysis and design activities
 Scheduling test implementation, execution and evaluation
 Finding and assigning resources for the different activities that have been defined
 Deciding what the documentation for the test project will be, e.g. which plans, how the test cases will be documented,
level of detail, etc.
 Defining the management information, including the metrics required and putting in place the processes to monitor and
control test preparation and execution, defect resolution and risk issues
 Ensuring that the test documentation generates repeatable test assets, e.g. test cases
 Budgeting for the test activities

Test Planning Entry Criteria Rendering
services
for:
Entry criteria are used to determine when a given test activity can start. This could include the beginning of a level of testing,
when test design and/or when test execution is ready to start.
Test environment available and ready for use (it functions)
Test tools installed in the environment are ready for use

Examples of
some typical
entry criteria to
test execution Testable code is available
may include
All test data is available and correct
Test design activity has completed

Test Planning Exit Criteria Rendering
services
for:
 Exit criteria are used to determine when a given test activity has been completed or when it should stop. Exit criteria can
be defined for all of the test activities, such as planning, specification and execution as a whole, or to a specific test level
for test specification as well as execution.
 Exit criteria should be included in the relevant test plans.
 Exit criteria should have been agreed as early as possible in the life cycle; however, they can be and often are subject
to controlled change as the detail of the project becomes better understood and therefore the ability to meet the criteria
is better understood by those responsible for delivery.

Rendering
Test Planning Exit Criteria services
for:
All tests planned have been run
A certain level of requirements coverage has been achieved

Some typical
exit criteria
might be:
No high-priority or severe defects are left outstanding
All high-risk areas have been fully tested, with only minor residual risks left
outstanding
The schedule has been achieved, e.g the release date has been reached and the
product has to go live
Cost-when the budget has been spent

Test Strategy, Test Approach Rendering
services
for:
Test Approach:
Test Strategy: The implementation of the test strategy for a specific
A high-level description of the test levels to be performed project. It typically includes the decisions made that follow
and the testing within those levels for an organization or based on the (test) project's goal and the risk assessment
programme (one or more projects). carried out, starting points regarding the test process, the
test design techniques to be applied, exit criteria and test
types to be performed.
One way to classify test approaches

or strategies is based on the point in
time at which the bulk of the test
design work is begun:
-Preventative approaches, where
tests are designed as early as
possible.
-Reactive approaches, where test
design comes after the software or
system has been produced.

services
for:
Analytical approaches Such as risk-based testing where testing is directed to areas of greatest risk
Such as stochastic testing using statistical information about failure rates (such as reliability growth
Model-based approaches models) or usage (such as operational profiles).
Such as failure-based (including error guessing and fault attacks), checklist based and quality-characteristic
Standard-compliant approaches based.
As specified by industry-specific standards such as The Railway Signaling standards (which define the
levels of testing required) or the MISRA (which defines how to design, build and test reliable software for
Methodical approaches the motor industry).
These adhere to the processes developed for use with the various agile methodologies or traditional
Process-compliant approaches waterfall approaches.

Rendering
Test Strategy, Test Approach services
for:
Major types of test strategies/ approach that are commonly found:
Such as exploratory testing where testing is more reactive to events than pre-planned, and
Dynamic and heuristic approaches where execution and evaluation are concurrent tasks.
Such as those where test coverage is driven primarily by the advice and guidance of technology
and/or business domain experts outside or within the test team.
Consultative approaches
Such as those that include reuse of existing test material, extensive automation of functional
Regression-averse approaches regression tests, and standard test suites.

services
for:
Factors to consider when selecting an approach:
•Risks: Risk management is very important during testing, so consider the risks and the level of risk. For a well-established
application that is evolving slowly, regression is an important risk, so regression-averse strategies make sense. For a new
application, a risk analysis may reveal different risks if you pick a risk-based analytical strategy.
•Skills: Consider which skills your testers possess and lack because strategies must not only be chosen, they must also be
executed. A standard compliant strategy is a smart choice when you lack the time and skills in your team to create your own
approach.
•Objectives: Testing must satisfy the needs and requirements of stakeholders to be successful. If the objective is to find as
many defects as possible with a minimal amount of up-front time and effort invested – for example, at a typical independent
test lab – then a dynamic strategy makes sense.

services
for:
•Regulations: Sometimes you must satisfy not only stakeholders, but also regulators. In this case, you may need to plan a
methodical test strategy that satisfies these regulators that you have met all their requirements.
•Product: Some products like, weapons systems and contract-development software tend to have well-specified
requirements. This leads to synergy with a requirements-based analytical strategy.
•Business: Business considerations and business continuity are often important. If you can use a legacy system as a
model for a new system, you can use a model-based strategy.

Test Execution Schedule Rendering
services
for:
Test Execution Schedule - A schedule for the execution of test suites within a test cycle.
Factors to take in consideration:

 Prioritization
 Dependencies
 Confirmation tests
 Regression tests
 Most efficient sequence for executing the tests
Ideally based on the prioritization -> Highest priority -> first to be run , but keep in mind all factors!
In case of various sequences are possible, with different level of efficiency -> trade-offs between efficiency and priority!

Test Execution Schedule – Exercise1 Rendering
services
for:
The following table shows 6 test procedures (P to U) that must now be entered into a test execution schedule.
Business Priority
Test Proce- (1 High Dependencies on test
Other dependencies
dure ID 2 Medium procedures
3 Low)
Can not start unless R has
P 1
completed
Q 1 None Regression testing only
R 2 None None
S 2 None None
Delivery of the code for this
T 3 None part of system is running very
late
U 3 None None

services
for:
 Business severity is regarded as the most important element in determining the sequence of the test procedures, but other
dependencies must also be taken into consideration.
 Regression testing can only be run once all other tests have completed.
Which of the following represents the MOST effective sequence for the test execution schedule (where the first entry in the
sequence is the first procedure to be run, the second entry is the second to be run and so on)?
A. Q, P, S, R, U, T.
B. R, S, U, P, Q, T.
C. R, P, S, U, T, Q.
D. P, Q, R, S, U, T

services
for:
As a Test Manager you have the following requirements to be tested:

Requirements to test:
R1 – Process Anomalies – High Complexity
R2 – Remote Services – Medium Complexity
R3 – Synchronization – Medium Complexity
R4 – Confirmation – Medium Complexity
R5 – Process closures – Low Complexity
R6 – Issues – Low Complexity
R7 – Financial Data – Low Complexity
R8 – Diagram Data – Low Complexity
R9 – Changes on user profile – Medium Complexity

services
for:
Requirements logical dependencies (A -> B means that B is dependent on A.:
218
BeeSpeed Timisoara Engineering Centre, 2017
services
for:
How would you structure the test execution schedule according to the
requirement dependencies?
A. R4 > R5 > R1 > R2 > R3 > R7 > R8 > R6 > R9.

B. R1 > R2 > R3 > R4 > R5 > R7 > R8 > R6 > R9.
C. R1 > R2 > R4 > R5 > R3 > R7 > R8 > R6 > R9.
D. R1 > R2 > R3 > R7 > R8 > R4 > R5 > R6 > R9.

Test Planning and Estimation Rendering
services
for:
The Metrics-Based Approach
This approach relies upon data collected from previous or similar projects. This kind of data might include:
 The number of test conditions
 The number of test cases written
 The number of test cases executed
 The time taken to develop test cases
 The time taken to run test cases
 The number of defects found
 The number of environment outages and how long on average each one lasted
With this approach and data it is possible to estimate quite accurately what the cost and time required for a similar project
would be.
It is important that the actual costs and time for testing are accurately recorded. These can then be used to re validate and
possibly update the metrics for use on the next similar project.

services
for:
The Expert-Based Approach
This alternative approach to metrics is to use the experience of owners of the relevant tasks or experts to derive an estimate
(this is also known as the Wide Band Delphi approach). In this context ‘experts’ could be:
 Business experts
 Test process consultants
 Developers
 Technical architects
 Analysts and designers
 Anyone with knowledge of the application to be tested or the tasks involved in the process.

services
for:
Many things affect the level of effort required to fulfill the test requirements of a project. These can be split into three main
categories, as shown below.
Product characteristics:
• Size of the test basis

• Complexity of the final product;
• The amount of non-functional requirements
• The security requirements (perhaps meeting BS 7799, the security standard)
• How much documentation is required (e.g. some legislation-driven changes demand a
certain level of
• documentation that may be more than an organization would normally produce)
• The availability and quality of the test basis (e.g. requirements and specifications).

services
for:
Development process characteristics
• Timescales
• Amount of budget available
• Skills of those involved in the testing and development activity (the lower the skill level in development, the more defects
could be introduced, and the lower the skill level in testing, the more detailed the test documentation needs to be)
• Which tools are being used across the life cycle (i.e. the amount of automated testing will affect the effort required
Expected outcome of testing such as
• The amount of errors

• Test cases to be written

Agenda Rendering
services
for:
5. Risk and Testing

Test Progress Monitoring and Control Rendering
services
for:
Test monitoring can serve various purposes during the project, including the following:
 Give the test team and the test manager feedback on how the testing work is going, allowing opportunities to guide and
improve the testing and the project.
 Provide the project team with visibility about the test results.
 Measure the status of the testing, test coverage and test items against the exit criteria to determine whether the test
work is done.
 Gather data for use in estimating future test efforts.
Information can be gathered manually, for small projects or automatically, when working with large teams, distributed
projects and long-term test efforts,

services
for:
Following are the Test coverage metric (requirements, user stories, acceptance criteria, risks or code)
metrics that can be
used for test
monitoring:
Percentage of planned work done in test environment and test cases preparation
Test execution metrics (Number of test cases pass, fail, blocked, on hold)
Defect metrics
Task completion, resources allocated and usage, effort
Cost of testing, including comparisons to the benefits of continuing testing

services
for:
Metrics should be collected during and at the end of a test level in order to assess:
 The adequacy of the test objectives for that test level
 The adequacy of the test approaches taken
 The effectiveness of the testing with respect to its objectives
 Progress against the planned schedule and budget

services
for:
Test reporting is concerned with summarizing information about the testing endeavor, including:
 What happened during a period of testing, such as dates when exit criteria were met
 Analyzed information and metrics to support recommendations and decisions about future actions, such as an
assessment of defects remaining, the economic benefit of continued testing, outstanding risks, and the level of
confidence in tested software. The outline of a test summary report is given in ‘Standard for Software Test
Documentation’ (IEEE 829)
 Factors impeding the progress
 The quality of the test object

services
for:
Typical test progress and test summary reports include:
 Summary of testing performed
 Information on what happened during a period of testing
 Deviations from plan (in schedule, duration, effort)
 Status of testing and product quality in respect with exit criteria/ definition of done
 Factors that have blocked or continue to block progress
 Metrics of defects, test cases, test coverage, activity progress and resource consumption
 Residual risks
 Reusable test work products produced

services
for:
Test control
describes any guiding or corrective actions taken as a result of information and metrics gathered
and reported. Actions may cover any test activity and may affect any other software life cycle activity or task.
Examples of test control actions are:
 Making decisions based on information from test monitoring.
 Re-prioritize tests when an identified risk occurs (e.g. software delivered late).
 Change the test schedule due to availability of a test environment.
 Set an entry criterion requiring fixes to have been retested (confirmation tested) by a developer before accepting them
into a build.

Agenda Rendering
services
for:
5. Risk and Testing

Configuration Management Rendering
services
for:
Configuration management establish and maintain the integrity of the products (components, data and documentation)
of the software/system through the project and product life cycle.
In Testing, Configuration Management must:

 Identify all test-ware items
 Establish and maintain the integrity of the testing deliverables (test plans, test cases, documentation) through the project
life cycle
 Set and maintain the version of these items
 Track the changes of these items
 Relate test-ware items to other software development items in order to maintain traceability
 Reference clearly all necessary documents in the test plans and test cases
Configuration management procedures have to be taken in account during test planning process.

Configuration Management Rendering
services
for:
Configuration Management activities:
Configuration Configuration
Configuration control Status accounting
identification auditing
selecting the configuration items for a system evaluation, co-ordination, approval or ->recording and reporting of The function to check
and recording their functional and physical disapproval, and implementation of changes to information needed to manage a that the software
characteristics in technical documentation configuration items after formal establishment configuration effectively, including: product matches the
of their configuration identification A listing of the approved configuration configuration items
identification identified previously
-The status of proposed changes to
the configuration
-The implementation status of the
approved changes

Agenda Rendering
services
for:
5. Risk and Testing

Risk and Testing Rendering
services
for:
In software testing Risks are the possible problems that might endanger the objectives of the project stakeholders.
It is the possibility of a negative or undesirable outcome. A risk is something that has not happened yet and it may never
happen; it is a potential problem.
In the future, a risk has some probability between 0% and 100%; it is a possibility, not a certainty.
The chance of a risk becoming an outcome is dependent on the level of risk associated with its possible negative
consequences.
We can classify risks into following categories:
 Product risk (factors relating to what is produced by the work, i.e. the thing we are testing)
 Project risk (factors relating to the way the work is carried out, i.e. the test project )

services
for:
Project Risks include(but not all):
Project issues Supplier issues Organizational factors Political issues Technical issues
Delays Failure of a third party Skill, training and staff Improper attitude Problems in defining
to deliver on time or at shortages toward or the right requirements
all expectations of testing
Inaccurate Contractual issues Personnel issues Follow up not done on Test environment not
estimations information found in ready in time
reviews
Late changes Conflicting business Communication Low quality of the
priorities issues design, code,
configuration data,
test data and tests
Weakness in the
process

services
for:
Potential failure areas (adverse future events or hazards) in software are known as product risks, as they are a risk to the
quality of the product.
Examples of product risks :

 Failure-prone software delivered
 The potential that a defect in the software/hardware could cause harm to an individual or company
 Poor software characteristics (e.g. functionality, security, reliability, usability, performance).
 Poor data integrity and quality (e.g. data migration issues, data conversion problems, data transport problems, violation
of data standards)
 Software that does not perform its intended functions
Risks are used to decide where to start testing in the software development life cycle and how much testing is needed.
Mitigating product risks may also involve non-test activities.

services
for:
Risk-based approach to testing provides proactive opportunities to reduce the levels of product risk starting in the initial
stages of a project. It involves the identification of product risks and how they are used to guide the test planning,
specification and execution.
In a risk approach the

risks identified Will determine the test techniques to be employed, and/or the extent of testing to be carried out
Prioritize testing in an attempt to find the critical defects as early as possible,
Will determine the particular test levels and types of testing to be performed
Will determine any non-test activities that could be employed to reduce risk

services
for:
To ensure that the chance of a product failure is minimized, risk management activities provide a disciplined approach
 To assess continuously what can go wrong (risks). Regular reviews of the existing and looking for any new product risks
should occur periodically throughout the life cycle
 To determine what risks are important to deal with (probability � impact). As the project progresses, owing to the
mitigation activities risks may reduce in importance, or disappear altogether
 To implement actions to deal with those risks (mitigating actions).
Testing is a risk control activity that provides feedback about the residual risk in the product by measuring the effectiveness
of critical defect removal and by reviewing the effectiveness of contingency plans.

Agenda Rendering
services
for:
5. Risk and Testing

Incident Management Rendering
services
for:
An incident is any unplanned event occurring that requires further investigation. In testing this translates into anything
where the actual result is different to the expected result.
An incident when investigated may be a defect, however, it may also be a change to a specification or an issue with the test
being run. It is important that a process exists to track all incidents through to closure.
The process of incident management ensures that incidents are tracked from recognition to correction, and finally through
retest and closure.
Incidents can be raised at any time throughout the software development life cycle, from reviews of the test basis
(requirements, specifications, etc.) to test specification and test execution.

services
for:
Incident reports have the following objectives:
To provide developers and other parties with feedback on the problem to enable
identification, isolation and correction as necessary
To provide test leaders with a means of tracking the quality of the system under test
and the progress of the testing
To provide ideas for test process improvement.

services
for:
An incident report should detail:

 Date of issue
 Organization and author
 Expected and actual results
 Identification of test item and environment involved
 Software or system life cycle process in which the incident was observed
 Description of the anomaly and how to reproduce it
 Scope or degree of impact on stakeholders interest
 Severity of the impact on the system, urgency/priority to fix
 Status of the incident
 Conclusions
 Recommendation, approvals, global issues
 Change history
 References
services
for:
According to the structure standardized by IEEE 829-1998, an incident report should include:
 Incident report id
 Summary
 Incident description with input
 Expected results, anomaly
 Date and time
 Procedure
 Environment
 Attempts to repeat
 Tester
 Observer
 Impact.

Agenda Rendering
services
for:
5. Risk and Testing

Check of Understanding Rendering
services
for:

Rendering
services
for:

– Tool Support for Testing

Agenda Rendering
services
for:
1. Types of Test Tools

2. Effective Use of Tools: Potential Benefits and Risks
3. Introducing a Tool into an Organization

Agenda Rendering
services
for:


Tool Support for Testing Rendering
services
for:
Test tool - A software product that supports one or more test activities, such as planning and control, specification,
building initial files and data, test execution and test analysis.
The main benefit of using test tools is similar to the main benefit of automating any process. That is, the amount of time and
effort spent performing routine, mundane, repetitive tasks is greatly reduced.
This time saved can be used to reduce the costs of testing or it can be used to allow testers to spend more time on the more
intellectual tasks

Types of Test Tools Rendering
services
for:
The test tools can be Their purpose

classified according
to:
The type of testing that they support
The source of tool/licensing model (shareware, open source, free or commercial)
The technology used
Who uses them

services
for:
The various types of test tools according to the test process activities are:
Tool support for Tool support for

Tool support for
management of test execution and
static testing:
testing and tests: logging
• Test management tools or • Review process support tools • Test design tools
application lifecycle • Static analysis tools (D) • Test data preparation tools
management tools (ALM) • Modelling tools (D) • Model based testing tools
• Requirements management • Acceptance test driven
tools development (ATDD) tools
• Defect management tools • Test driven development
• Configuration management (TDD) tools
tools
• Continuous integration tools
(D)
services
for:
Tool support for test Tools support for Tool support for
execution and performance and specialized testing
logging monitoring:
needs:
• Test execution tools • Dynamic analysis tools • Data quality assessment
• Test harness/ Unit test (D) • Data conversion and

framework tools (D) • Performance testing, migration
• Test comparators Load testing and stress- • Usability testing
• Coverage measurement testing tools • Accessibility testing

tools (D) • Monitoring tools • Localization testing
• Security tools • Security testing
• Portability testing

Types of Test Tools -Test Rendering
services
management tools for:
Requirement
management
Tool
Continuous Configuration
Integration Management
Tool(D)
Test Tool
Management
Tool
Test
Defect management
Management and ALM
Tool
Tool

Types of Test Tools -Test Rendering
services
management tools for:
Features or Interfaces to other tools or spreadsheets

characteristics of test
management tools are:
Produce useful information in a format that fits the needs
Maintain traceability to requirements in a requirements management tool
To link the test object with version information in the configuration management tool
!When using an integrated tool, as ALM tool, keep in mind that test management tool is only a
module and it acts as an interfaces with other tools and modules!

Types of Test Tools – Test execution Rendering
services
tools for:
Data-driven testing - A scripting technique that stores test input and expected results in a table or spreadsheet, so
that a single control script can execute all of the tests in the table. Data-driven testing is often used to support the application
of test execution tools such as capture/playback tools.
Keyword-driven testing - A scripting technique that uses data files to contain not only test data and expected
results, but also keywords related to the application being tested. The keywords are interpreted by special supporting scripts
that are called by the control script for the test .
Both approaches need someone with expertise in the scripting language.
In both approaches the expected results need to be compared with actual results either dynamically or stored for later
comparison

Types of Test Tools – Test execution Rendering
services
tools for:
Model-based Testing (MBT)- A scripting technique that enables a functional specification to be captures in form
of a model, such as an activity diagram.
Generally performed by a system designer.
The MBT tool interprets the model in order to create test case specifications.

Agenda Rendering
services
for:


3. Effective Use of Tools: Rendering
services
Potential Benefits for:
There are many benefits that can be gained by using tools to support testing. They are:
 Reduction of repetitive work

 Greater consistency and repeatability
 Objective assessment
 Ease of access to information about tests or testing (e.g. statistics and graphs about test progress, incident rates and
performance).

Effective Use of Tools: Risks Rendering
services
for:
Risks include: Unrealistic expectations from the tool
People often make mistakes by underestimating the time, cost and effort for the initial introduction of a tool
Underestimating the time and effort needed to achieve significant and continuing benefits from the tool (including the
need for changes in the testing process and continuous improvement of the way the tool is used
Underestimating the effort required to maintain the test assets generated by the tool.
Over-reliance on the tool
Lack of a dedicated test automation specialist
Lack of good understanding and experience with the issues of test automation
Lack of stakeholders commitment for the implementation of a such tool

Effective Use of Tools: Risks Rendering
services
for:
Risks include: Version control of test assets may be neglected
Relationships and interoperability issues between critical tools
The tool vendor may go out of business
The vendor nay provide poor response for support, upgrades and defect fixes
An open sources project may be suspended
A new platform or technology may not be supported by the tool

Agenda Rendering
services
for:


4. Introducing a Tool into an Rendering
services
Organization for:
The following factors Assessment of the organization’s maturity (e.g. readiness for change);
are important during
tool selection:
Identification of the areas within the organization where tool support will help to improve testing
processes;
Evaluation of tools against clear requirements and objective criteria;
Proof-of-concept to see whether the product works as desired and meets the requirements and
objectives defined for it;
Evaluation of the vendor (training, support and other commercial aspects) or open-source
network of support;
Evaluation of training needs
Estimation of a cost-benefit ratio based on a concrete business case

4. Introducing a Tool into an Rendering
services
Organization for:
The following factors Understanding the technologies used by the test object in order to select a compatible tool
are important during
tool selection:
The build and continuous integration tools already in use in the organization, in order to assure
tool compatibility and integration
Availability of the tool for a free trial period
Coaching and mentoring in the use of tools
Pro and Cons of various licensing models

Introducing a Tool into an Rendering
services
Organization for:
The objectives for a

To learn more about the tool and in detail.
pilot project for a new
tool are:
To see how the tool would fit with existing processes or documentation, how those would need to change to work well with
the tool and how to use the tool to streamline existing processes;
To decide on standard ways of using the tool that will work for all potential users (e.g. naming conventions, creation of
libraries, defining modularity, where different elements will be stored, how they and the tool itself will be maintained);
To evaluate the pilot project against its objectives (have the benefits been achieved at reasonable cost?).
Understanding the metrics that you wish the tool to collect and report, and configuration of the tool in order to capture
these metrics

Introducing a Tool into an Rendering
services
Organization for:
Success factors like: Rolling out the tool to the rest of the organization incrementally.
Adapting and improving processes to fit in with the use of the tool.
Providing training and coaching/mentoring for new users.
Defining usage guidelines.
Implementing a way to gather usage information from the actual use of a given tool.
Monitoring tool use and benefits.
Providing support for the test team for a given tool.
Gathering lessons learned from all teams

Agenda Rendering
services
for:


Check of Understanding Rendering
services
for:

ISTQB Foundations of Software Testing - Fundamentals of Testing

Uploaded by

Document Information

Original Title

Copyright

Available Formats

Share this document

Share or Embed Document

Sharing Options

Did you find this document useful?

Is this content inappropriate?

Copyright:

Available Formats

ISTQB Foundations of Software Testing - Fundamentals of Testing

Uploaded by

Copyright:

Available Formats

04 –

1 BeeSpeed Timisoara Engineering Centre, 2017

1. What testing is and what testing does?

2 BeeSpeed Timisoara Engineering Centre, 2017

1. What testing is and what testing does?

3 BeeSpeed Timisoara Engineering Centre, 2017

Testing is an activity to reduce risk and improve quality by finding defects.

 Testing and debugging

4 BeeSpeed Timisoara Engineering Centre, 2017

What testing is and what testing does?

 Static testing and Dynamic testing

- Static testing is where the code is not exercised (e.g Reviews)

5 BeeSpeed Timisoara Engineering Centre, 2017

What testing is and what testing does?

The main goals of testing:

What testing is and what testing does?

Different viewpoints and objectives:

 Component testing – find as many defects as possible/ increase code coverage

7 BeeSpeed Timisoara Engineering Centre, 2017

1. What testing is and what testing does?

8 BeeSpeed Timisoara Engineering Centre, 2017

- Testing’s Contribution to Success

 Examples of Testers contribution within the software development lifecycle:

9 BeeSpeed Timisoara Engineering Centre, 2017

- Why do software have bugs and defects?

 An incorrect software can harm everything

 The effect of software failures lead to

10 BeeSpeed Timisoara Engineering Centre, 2017

- Why do software have bugs and defects?

■ Online British taxation

11 BeeSpeed Timisoara Engineering Centre, 2017

-Error, Defect, Failure

 Who can make an error or mistake?

Error / Mistake Defect / Bug Failure

-False positive – False negative

13 BeeSpeed Timisoara Engineering Centre, 2017

Defects, Root Causes and Effects

14 BeeSpeed Timisoara Engineering Centre, 2017

- Testing Against Errors

 If we wish to influence errors

 Through the whole development Software Development Life Cycle (SDLC)

15 BeeSpeed Timisoara Engineering Centre, 2017

 Results of the failure:

 It is necessary to start bug searching ASAP:

 Because of inadequate testing!

 „We can be rather good if we aim for perfection”

17 BeeSpeed Timisoara Engineering Centre, 2017

- Numbers, Numbers, Numbers

$ 100,000 >> Cost of 1 hour unscheduled halt

10,000 x >> Cost of live environment correction

80% >> Unplanned halts could be prevented with

75% >> Application is introduced without testing

18 BeeSpeed Timisoara Engineering Centre, 2017

- Can we test everything?

- There is a system with 20 independent inputs, each has 4 possible values

 The aim is the better efficiency of testing

19 BeeSpeed Timisoara Engineering Centre, 2017

- Testing and Risks

 Risk is inherit in all software development

20 BeeSpeed Timisoara Engineering Centre, 2017

- Testing and Quality

21 BeeSpeed Timisoara Engineering Centre, 2017