Ses4 Test Levels

Software Testing Foundation
Test Levels
9/21/2022 09e-BM/DT/FSOFT - ©FPT SOFTWARE – Fresher Academy - Internal Use 1

Lesson Objectives
 Be introduced test levels (UT, IT, ST, UAT)
 Have chance to discuss with experience testers

AGENDA
• Test Levels
• Unit Test (UT)
• Integration Test (IT)
• System Test (ST)
• User Acceptance Test (UAT)

Section 1
TEST LEVELS

Test Levels – What?
 A group of test activities that are organized and managed

together.
 Each test level is specifically service for a development
stage -> V-model

V-Model

Test Levels
Test levels are characterized by the following attributes:

 Specific objectives
 Test basis, referenced to derive test cases
 Test object (i.e., what is being tested)
 Typical defects and failures
 Specific approaches and responsibilities

Test Levels
 Unit testing (Component testing)
 Integration testing (Combination testing)
 System testing
 Acceptance testing (User Acceptance testing)

Section 2
UNIT TEST

Unit Test – What?
 Verify that individual UNITs of software program are working

properly follow detailed design
 A UNIT(component) is the smallest testable part of an application,

it can be:
Program statements
Class, Function, Procedure
Module
Individual program
Unit Test – Why?
$14,000
 Faster Debugging
85% % Defects
 Better Design
Percentage of Bugs
Introduced in
this phase
 Reduce Future % Defects
Cost (Quality found in
in this phase
$1000
Effort & $ Cost to
Correction Cost) $25 $130 $250 repair defect
in this phase
Coding Unit Funct Field Post
Test Test Test Release

Unit Test – Inputs (Test Basic)
 Detailed design
 Code
 Data model
 Component specifications

Unit Test – Implementation
 Doer: developer/coder, who develop the code

 How: using one or more unit testing types in following orders:
 Top-Down Unit Testing: stubs are required
 Bottom-up Unit Testing: drivers are required
Stubs and drivers are usually used in Unit Testing to replace
missing components, software

Top-Down Unit Testing
Stubs: called by software component
Stub
Stub
B
Stub
Bottom-up Unit Testing
Driver: call software component
Test Test
Drivers Drivers
Level N Level N
Test
Drivers
Level N-1
Unit Test – Types/Technique
Statement Testing: execute all statements at least

once
Decision/Branch Testing: execute each
decision/branch at least once
Path Testing: execute all paths in the program at least
once

Statement Testing
Execute every line of code (mean: all statements in the

programs should be executed at least once)
Example:
READ A
READ B
C =A + 2*B
IF C> 50 THEN
PRINT “large C”
ENDIF
=> 100% statement coverage: A = 20, B = 25
Decision (Branch) Testing
Execute every branches (all branches in the program should be executed at
least once). Example:
z = 0;
if (a > b) then
z = 12;
x = 72 / z
Statement coverage may miss bugs
One test needed for statement coverage
(a = 5, b = 4)  Passes
Select inputs to force each decision to execute both possible ways (T/F)
Now two test cases are needed for coverage
(a = 5, b = 4)  Passes
(a = 4, b = 5)  Finds bug
Path Testing
Is combination between statement and P1

decision testing T F
Paths in the program executed at least

once. P2
T F
Independent path can be defined in
terms of CFG – Control Flow Graph.
Test the each independent path.
Path Testing
READ A (1)
READ B (2)
C = A + 2 * B (3)
IF C> 50 THEN (4)
PRINT “large C” (5)
ENDIF (6)
Path coverage:
Path 1: 1, 2, 3, 4, 5, 6
A = 20, B = 25
Path 2: 1, 2, 3, 4, 6
A = 20, B= 10
Path Testing
Read P, Q (1)
If P+Q > 100 then (2)
Print "Large“ (3)
If P > 100 then (4)
Print "P is Greater“ (5)
Endif (6)
Path coverage:
Path 1: 1, 2, 3, 4, 5, 6
P= 110, Q = 50
Path 2: 1, 2, 3, 6
P = 80, Q = 70
Path 3: 1, 2, 6
P=80, Q=10
Unit Test – Typical defects and failures
Incorrect functionality (e.g., not as described in

design specifications)
Data flow problems
Incorrect code and logic

Unit Test – Tools
 Java: Junit (www.junit.org)

 C/C++: cppUnit
(http://sourceforge.net/projects/cppunit/)
 Python: pyUnit (http://pyunit.sourceforge.net/)
 Perl: PerlUnit (http://perlunit.sourceforge.net/)
 Visual Basic: vbUnit (http://www.vbunit.com/)
 C#: csUnit (http://www.csunit.org/)
 .NET: Nunit (http://www.nunit.org/)
Section 3
INTEGRATION TEST

Integration Testing – What?
Integration testing focuses on interactions between

components or systems
Notes: Test the interface between components or

system and do not concentrate on functionalities of
each function itself.

Integration Testing – Why, When?
Why:
Unit tests only test the unit in isolation
Many failures result from faults in the interaction of
subsystems
Ensure integrate functions/modules follow architecture
design or high level design (HLD)
Ensure communication between functions/modules work
properly
When:
Depends on integration level
Integration Testing – Inputs (Test Basis)
 Software and system/architecture design

 Sequence diagrams
 Interface and communication protocol
specifications
 Use cases
 Architecture at component or system level
 Workflows
 External interface definitions
Level of Integration
Components integration: integrate between

components.
Ex: integrate create new, cancel and update order.
Systems integration: integrate between systems,
packages, and microservices.
Ex: integrate “Test Insight” system and “Defect
management” system.

Component IT – Typical defects and failures
 Incorrect data, missing data, or incorrect data encoding

 Incorrect sequencing or timing of interface calls
 Interface mismatch
 Failures in communication between components
 Unhandled or improperly handled communication
failures between components
 Incorrect assumptions about the meaning, units, or
boundaries of the data being passed between
components

System IT – Typical defects and failures
 Inconsistent message structures between systems

 Incorrect data, missing data, or incorrect data encoding
 Interface mismatch
 Failures in communication between systems
 Unhandled or improperly handled communication
failures between systems
 Incorrect assumptions about the meaning, units, or
boundaries of the data being passed between systems
 Failure to comply with mandatory security regulations

Types of Integration
Big-bang integration: integrate all

components, systems at one time.
Incremental integration: integrate one
by one component

Big Bang Integration
Test A
A
Test B
Test C B C D
Test
Test D A, B, C, D,
E, F, G
Test E E F G
Test F
Integrate all components, systems at
Test G
one time
Big Bang Integration
Advantages:
 Everything is finished before integration testing starts.
Disadvantages:
 Time consuming
 Difficult to trace cause of fault
 Only effectively when the software expected to have no
defect.

Incremental Integration - Types
Top-down: from menu to function (use

stub)
Bottom-up: from detail function to menu
(use driver)
Functional incremental: each function is
integrated, tested one by one.

Top-down Integration Testing
A
Layer I
B C D Layer II
E F G
Layer III
Test
Test A Test A, B, C, D A, B, C, D,
E, F, G
Layer I
Layer I + II
All Layers
Top-down Integration Testing
Advantages:
No drivers needed
Test cases can be defined in terms of the
functionality of the system (functional requirements)
Disadvantages:
Large number of stubs may be required, especially
if the lowest level of the system contains many
methods.
Writing stubs is difficult: Stubs must allow all
possible conditions to be tested.
Some interfaces are not tested separately.
Bottom-Up Integration Testing
A
Layer I
B C D Layer II
Test E
Test B, E, F E F G
Layer III
Test F
Test C Test
A, B, C, D,
E, F, G
Test D,G
Test G
Bottom-Up Integration Testing
Advantages:
No stubs needed
Useful for integration testing of the following systems
•Object-oriented systems
•Real-time systems
•Systems with strict performance requirement
Disadvantages:
Tests the most important subsystem (user
interface) last
Drivers needed
Functional Incremental
Advantages:
Test cases can be defined in terms of the
functionality of the system (functional requirements)
Disadvantages:
Both stubs and drivers are needed.
Some interfaces are not tested separately.
Difficulty for integrator
Section 4
SYSTEM TEST

System Testing – What?
System testing is defined as testing the behavior of a

system/software as per software requirement
specification.
Is concerned with the behavior of the whole system or

product as defined by the scope of a development
project or product, included functional requirements
and non-functional requirements.

System Testing – Why?
It is done to verify, validate the functional, non

functional, business, technical requirements of the
software.
Is testing in an environment that closely resembles the

production environment where the application will be
finally deployed
It is the final test on behalf of development to verify

that the system 09e-BM/DT/FSOFT
9/21/2022
to be delivered
- ©FPT SOFTWARE – Fresher Academy - Internal Use 42
System Testing – When?
Complete software system should be developed

Unit testing must be completed
Integration testing must be completed
Specifications for the product have been completed
Test cases, test data and test schedule are ready

System Testing – Techniques
System testing includes:

Functional testing: using back box test method
Non-functional testing: performance and reliability.
Re-testing and Regression testing
 Doer: Tester

Black Box Test Method
 Focuses on input and output of the software without regard to the internal
code of the program
 Focuses on functional requirement of the software
 Design sets of input conditions that will fully exercise all functional
requirements for a program

System Testing – Environment
Environment for system test should be the

most likely/closely to the production
environment to minimize environment-specific
failure.
Production environment is where the
software/system/application will be finally
deployed
System Testing – Types
Functional Testing types

Function Testing
User Interface (UI) Testing
Data and Database Integrity Testing
Business Cycle Testing
Access Control Testing
System Testing – Types
Non-functional Testing types

Performance testing
Load testing Testing related to changes
Stress testing types
Volume testing Confirmation testing
Regression testing

System Testing – Typical defects and failures
 Incorrect calculations
 Incorrect or unexpected system functional or non-functional behavior
 Incorrect control and/or data flows within the system
 Failure to properly and completely carry out end-to-end functional tasks
 Failure of the system to work properly in the production environment(s)
 Failure of the system to work as described in system and user manuals

Section 5
(USER ) ACCEPTANCE TEST

Acceptance Testing – What?
Also known as user acceptance testing

Do by customer.
Is the last phase of the software testing process
Confirm that the system meets the agreed upon criteria
Is a user-run test that demonstrates the application’s
ability to meet user’s requirements (mapping to V-model)
Concentrates on validation types of testing to determine
whether the system is fit for use.

Acceptance Testing – Why?
Accept product based on acceptance criteria

Ensure function needed and expected by customers is
present in a software product

Acceptance Testing – Inputs
 Business processes
 User or business requirements
 Regulations, legal contracts and standards
 Use cases
 System requirements
 System or user documentation
 Installation procedures
 Risk analysis reports

Acceptance Testing – When?
After software product is released and

System testing is implemented
(Note: time scale of AT may overlap on ST – alpha

testing)

Acceptance Testing – Environment
For most aspect should represent for the production

environment.

Acceptance Testing – Types
 Operational Acceptance Testing – OAT: ensures that all the

non-functional aspects of a system are tested to ensure that
the system is within the specified parameters. It may includes:
• Backup & restore
• Installing, uninstalling and upgrading
• Disaster recovery
• User management
• Maintenance tasks
• Data load and migration tasks
• Security weakness
• Performance testing

Acceptance Testing – Types
 Compliance Acceptance Testing: is performed against the

contract’s acceptance criteria for producing custom-developed
software, such as government regulations.
Eg: does the accounting report adhere to the law?
 Mass market Acceptance Testing contains 2 phases:

Alpha testing: do by users in development site.
Beta testing: do by users in customer site.

Alpha Testing
Do by end users/customers
Conducted at the developer’s site / development site
Software is used in a natural setting with developers

watching intently (under the supervision of the
developer)

Beta Testing
Do by end-users/customers
Conducted at end-user sites
Do after alpha testing
Developer is generally not present
The end-user records all problems that are
encountered and reports these to the developers at
regular intervals

AT – Typical defects and failures
System workflows do not meet business or user requirements

Business rules are not implemented correctly
System does not satisfy contractual or regulatory requirements
Non-functional failures such as security vulnerabilities,
inadequate performance efficiency under high loads, or improper
operation on a supported platform

References
 https://www.istqb.org/ , Foundations of Software Testing,
Chapter 2

Lesson Summary
• Session 1: Test Levels
• Session 2: Unit Test (UT)
• Session 3: Integration Test (IT)
• Session 4: System Test (ST)
• Session 5: User Acceptance Test (UAT)

Thank you

Ses4 Test Levels

Uploaded by

Document Information

Copyright

Available Formats

Share this document

Share or Embed Document

Sharing Options

Did you find this document useful?

Is this content inappropriate?

Copyright:

Available Formats

Ses4 Test Levels

Uploaded by

Copyright:

Available Formats

Software Testing Foundation

9/21/2022 09e-BM/DT/FSOFT - ©FPT SOFTWARE – Fresher Academy - Internal Use 1

 Be introduced test levels (UT, IT, ST, UAT)

 Have chance to discuss with experience testers

9/21/2022 09e-BM/DT/FSOFT - ©FPT SOFTWARE – Fresher Academy - Internal Use 2

• Unit Test (UT)

• Integration Test (IT)

• System Test (ST)

• User Acceptance Test (UAT)

9/21/2022 09e-BM/DT/FSOFT - ©FPT SOFTWARE – Fresher Academy - Internal Use 3

9/21/2022 09e-BM/DT/FSOFT - ©FPT SOFTWARE – Fresher Academy - Internal Use 4

 A group of test activities that are organized and managed

9/21/2022 09e-BM/DT/FSOFT - ©FPT SOFTWARE – Fresher Academy - Internal Use 5

9/21/2022 09e-BM/DT/FSOFT - ©FPT SOFTWARE – Fresher Academy - Internal Use 6

Test levels are characterized by the following attributes:

 Test basis, referenced to derive test cases

 Test object (i.e., what is being tested)

 Typical defects and failures

 Specific approaches and responsibilities

9/21/2022 09e-BM/DT/FSOFT - ©FPT SOFTWARE – Fresher Academy - Internal Use 7

 Unit testing (Component testing)

 Integration testing (Combination testing)

 Acceptance testing (User Acceptance testing)

9/21/2022 09e-BM/DT/FSOFT - ©FPT SOFTWARE – Fresher Academy - Internal Use 8

9/21/2022 09e-BM/DT/FSOFT - ©FPT SOFTWARE – Fresher Academy - Internal Use 9

 Verify that individual UNITs of software program are working

 A UNIT(component) is the smallest testable part of an application,

9/21/2022 09e-BM/DT/FSOFT - ©FPT SOFTWARE – Fresher Academy - Internal Use 11

9/21/2022 09e-BM/DT/FSOFT - ©FPT SOFTWARE – Fresher Academy - Internal Use 12

 Doer: developer/coder, who develop the code

9/21/2022 09e-BM/DT/FSOFT - ©FPT SOFTWARE – Fresher Academy - Internal Use 13

Statement Testing: execute all statements at least

9/21/2022 09e-BM/DT/FSOFT - ©FPT SOFTWARE – Fresher Academy - Internal Use 16

Execute every line of code (mean: all statements in the

Is combination between statement and P1

Paths in the program executed at least

Incorrect functionality (e.g., not as described in

9/21/2022 09e-BM/DT/FSOFT - ©FPT SOFTWARE – Fresher Academy - Internal Use 22

 Java: Junit (www.junit.org)

9/21/2022 09e-BM/DT/FSOFT - ©FPT SOFTWARE – Fresher Academy - Internal Use 24

Integration testing focuses on interactions between

Notes: Test the interface between components or

9/21/2022 09e-BM/DT/FSOFT - ©FPT SOFTWARE – Fresher Academy - Internal Use 25

 Software and system/architecture design

Components integration: integrate between

9/21/2022 09e-BM/DT/FSOFT - ©FPT SOFTWARE – Fresher Academy - Internal Use 28

 Incorrect data, missing data, or incorrect data encoding

9/21/2022 09e-BM/DT/FSOFT - ©FPT SOFTWARE – Fresher Academy - Internal Use 29

 Inconsistent message structures between systems

9/21/2022 09e-BM/DT/FSOFT - ©FPT SOFTWARE – Fresher Academy - Internal Use 30

Big-bang integration: integrate all

9/21/2022 09e-BM/DT/FSOFT - ©FPT SOFTWARE – Fresher Academy - Internal Use 31

9/21/2022 09e-BM/DT/FSOFT - ©FPT SOFTWARE – Fresher Academy - Internal Use 33

Top-down: from menu to function (use

9/21/2022 09e-BM/DT/FSOFT - ©FPT SOFTWARE – Fresher Academy - Internal Use 34

9/21/2022 09e-BM/DT/FSOFT - ©FPT SOFTWARE – Fresher Academy - Internal Use 40

System testing is defined as testing the behavior of a

Is concerned with the behavior of the whole system or

9/21/2022 09e-BM/DT/FSOFT - ©FPT SOFTWARE – Fresher Academy - Internal Use 41

It is done to verify, validate the functional, non

Is testing in an environment that closely resembles the

It is the final test on behalf of development to verify

Complete software system should be developed

9/21/2022 09e-BM/DT/FSOFT - ©FPT SOFTWARE – Fresher Academy - Internal Use 43