Levels Testing Integration

Different Levels of Testing
System
Unit Testing Functional Testing Component Testing Testing
Program unit A
Function 1
Program unit B
Component 1
. . Whole
. Function 2 System
. .
. (e.g.
regression
Component 3
testing)
Function 8
Program unit T
Structural (white-box)
Bernd Bruegge & Allen H. Dutoit testing more
Object-Oriented Software functional
Engineering: (Black-box)
Using UML, Patterns, and Java testing
1
Integration Testing
• The entire system is viewed as a collection of

subsystems (sets of classes) determined during
the system and object design
• Goal: Test all interfaces between subsystems

and the interaction of subsystems
• The Integration testing strategy determines the

order in which the subsystems are selected for
testing and integration.
Bernd Bruegge & Allen H. Dutoit Object-Oriented Software Engineering: Using UML, Patterns, and Java 2
Why Do You Need Integration Testing?
• To make sure that

your components
satisfy the
fallowing
requirements:
• Functional.
• Performance.
• Reliability.
Bernd Bruegge & Allen H. Dutoit Object-Oriented Software Engineering: Using UML, Patterns, and Java 3 3
Why do we do integration testing?
• Unit tests only test the unit in isolation
• Many failures result from faults in the interaction of

subsystems
• Often many Off-the-shelf components are used that

cannot be unit tested
• Without integration testing the system test will be very

time consuming
• Failures that are not discovered in integration testing will

be discovered after the system is deployed and can be
very expensive.
Stubs and drivers
• Driver:
Driver
• A component, that calls the TestedUnit
• Controls the test cases
Tested
• Stub: Unit
• A component, the TestedUnit
depends on
• Partial implementation
Stub
• Returns fake values.
Stubs
• stub: A controllable replacement for an existing

software unit to which your code under test has a
dependency.
• useful for simulating difficult-to-control elements:

• network / internet
• database
• time/date-sensitive code
• files
• threads
• memory
• also useful when dealing with brittle legacy code/systems
Create a stub, step 1
• Identify the external dependency.

• This is either a resource or a class/object.
• If it isn't an object, wrap it up into one.
• (Suppose that Class A depends on troublesome
Class B.)
• Extract the core functionality of the object into

an interface.
• Create an InterfaceB based on B
• Change all of A's code to work with type InterfaceB,
not B
• Write a second "stub" class that also implements the

interface,
but returns pre-determined fake data.
• Now A's dependency on B is dodged and can be tested easily.
• Can focus on how well A integrates with B's external
behavior.
"Mock" objects
• mock object: A fake object that decides

whether a unit test has passed or failed by
watching interactions between objects.
• useful for interaction testing (as opposed to state
testing)
Stubs vs. mocks
• A stub gives out data that goes to
the object/class under test.
• The unit test directly asserts against
class under test, to make sure it gives
the right result when fed this data.
• A mock waits to be called by

the class under test (A).
• Maybe it has several methods
it expects that A should call.
• It makes sure that it was contacted
in exactly the right way.
• If A interacts with B the way it should,
the test passes.
Example: A 3-Layer-Design (Spreadsheet)
A
Spread
A
SheetView Layer I
B C D
Entity
Data Currency
B Calculator
C D Layer II
Model Converter
E F G
BinaryFile XMLFile Currency Layer III
E F G
Storage Storage DataBase
Big-Bang Approach A
B C D
Test A
Test B
E F G
Test C
Test
Test D A, B, C, D,
E, F, G
Test E
Test F
Test G
Bottom-up Testing Strategy
• The subsystems in the lowest layer of the call
hierarchy are tested individually
• Then the next subsystems are tested that call the
previously tested subsystems
• This is repeated until all subsystems are included
• Drivers are needed.
Bottom-up Integration A
B C D
Test E
E F G
Test B, E, F
Test F
Test C Test
A, B, C, D,
E, F, G
Test G Test D,G
Pros and Cons of Bottom-Up Integration Testing
• Con:
• Tests the most important subsystem (user interface)
last
• Drivers needed
• Pro
• No stubs needed
• Useful for integration testing of the following systems
• Object-oriented systems
• Real-time systems
• Systems with strict performance requirements.
Top-down Testing Strategy
• Test the top layer or the controlling subsystem

first
• Then combine all the subsystems that are called
by the tested subsystems and test the resulting
collection of subsystems
• Do this until all subsystems are incorporated
into the test
• Stubs are needed to do the testing.
Top-down Integration A
B C D
E F G
Test
Test A Test A, B, C, D A, B, C, D,
E, F, G
Layer I Layer I + II All Layers
Pros and Cons of Top-down Integration Testing
Pro
• Test cases can be defined in terms of the
functionality of the system (functional
requirements)
• No drivers needed
Cons
• Writing stubs is difficult: Stubs must allow all
possible conditions to be tested.
• Large number of stubs may be required,
especially if the lowest level of the system
contains many methods.
• Some interfaces are not tested separately.
Sandwich Testing Strategy
• Combines top-down strategy with bottom-up

strategy
• The system is viewed as having three layers
• A target layer in the middle
• A layer above the target
• A layer below the target
• Testing converges at the target layer.
Sandwich Testing Strategy A
B C D
Test A
E F G
Test A,B,C, D
Test E
Test
Test B, E, F A, B, C, D,
Test F E, F, G
Test D,G
Test G
Pros and Cons of Sandwich Testing
• Top and Bottom Layer Tests can be done in

parallel
• Problem: Does not test the individual

subsystems and their interfaces thoroughly
before integration
• Solution: Modified sandwich testing strategy
Modified Sandwich Testing Strategy
• Test in parallel:
• Middle layer with drivers and stubs
• Top layer with stubs
• Bottom layer with drivers
• Test in parallel:
• Top layer accessing middle layer (top layer
replaces drivers)
• Bottom accessed by middle layer (bottom
layer replaces stubs).
Modified Sandwich Testing A
B C D
Test A
Test A,C
Test C E F G
Test B
Test
Test E Test B, E, F A, B, C, D,
E, F, G
Test F
Test D
Test D,G
Test G
Continuous Testing
• Continuous build:
• Build from day one
• Test from day one
• Integrate from day one
 System is always runnable
• Requires integrated tool support:

• Continuous build server
• Automated tests with high coverage
• Tool supported refactoring
• Software configuration management
• Issue tracking.
Continuous Testing Strategy
A
Spread
SheetView Layer I
B C D
Data Currency
Calculator Layer II
Model Converter
E F G
BinaryFile XMLFile Currency Layer III
Storage Storage DataBase
+ Cells
Sheet View + File Storage
+ Addition
Steps in Integration Testing
1.
1. Based
Based onon the
the integration
integration 4.
4. Test
Test subsystem
subsystem
strategy, decomposition:
decomposition: Define
Define test
strategy, select
select aa cases that exercise all
test
component
component to to be
be tested.
tested. cases that exercise all
Unit dependencies
dependencies
Unit test
test all
all the
the classes
classes in
in
. the 5.
5. Test
Test non-functional
the component.
component. non-functional
requirements:
2. requirements: Execute
Execute
2. Put
Put selected
selected component
component performance
performance tests
tests
together;
together; do do anyany 6.
6. Keep
Keep records
records of
of the
the test
test
preliminary
preliminary fix-up
fix-up cases and testing activities.
necessary cases and testing activities.
necessary to make
to make the
the 7.
integration
integration testtest operational
operational 7. Repeat
Repeat steps
steps 11 toto 77 until
until
the full system is tested.
the full system is tested.
(drivers,
(drivers, stubs)
stubs)
3.
3. Test
Test functional
functional
requirements: The
The primary
primary goal goal of
of integration
integration
requirements: Define Define test
test testing
cases testing isis to
to identify
identify failures
failures
cases that
that exercise
exercise allall uses
uses with the (current)
with the (current)
cases
cases with
with the
the selected
selected component
component component configuration.
configuration.
component
Daily builds
• daily build: Compile working executable on a daily basis

• allows you to test the quality of your integration so far
• helps morale; product "works every day"; visible progress
• best done automated or through an easy script
• quickly catches/exposes any bug that breaks the build
• smoke test: A quick set of tests run on the daily build.

• NOT exhaustive; just sees whether code "smokes" (breaks)
• used (along with compilation) to make sure daily build runs
• continuous integration:
Adding new units immediately as they are written.
Some Metrics for Integration Test
based on Structural Decomposition
• For Top-down approach with n nodes,

there is a potential need to construct as
much as (n-1) stubs.
• For Bottom-up approach with n nodes and

v leaves, there is a potential need to
construct as much as (n- v) drivers.
• For both cases there may be as much as

(n – v + edges) number of test
sessions (e.g. cases or scenarios)
Simple Construct Example for the Metrics
B C D E
• There are 5 nodes and 4 leaves and 4

edges: Number of test sessions = 5:
1. n- 1 = 5 -1 = 4 stubs (Top-down) (Bottom-up)

2. n - v = 5 - 4 = 1 driver 1. A is complete: E is complete:
3. n - v+ edges = 5 - 4 + 4 = 5 test test with 4 stubs test with driver A
sessions
2. B is complete: D is complete:
test with 3 stubs test with driver A
3. C is complete: C is complete:
4. D is complete: B is complete:
test with 1 stub test with driver A
5. E is complete: A is complete:
Bernd Bruegge & Allen H. Dutoit test allUsing
Object-Oriented Software Engineering: modules test all modules
UML, Patterns, and Java 30
Slightly Modified Example for the Metrics
A
B Number of test sessions = 6:

E
C (Top-down) (Bottom-up)
D 1. A is complete: E is complete:
• There are 5 nodes and 3 leaves and 4 2. B is complete: D is complete:
edges: test with 2 stubs test with driver B
1. n- 1 = 5 -1 = 4 stubs 3. B is complete: C is complete:
2. n - v = 5 - 3 = 2 drivers test with A test with driver B
3. n - v+ edges = 5 – 3 + 4 = 6 test
sessions 4. C is complete: B is complete:
test with B test with C and D
5. D is complete: B is complete:
test with B test with driver A
6. E is complete: A is complete:
Bernd Bruegge & Allen H. Dutoit
test all modules test all modules
Object-Oriented Software Engineering: Using UML, Patterns, and Java 31

Levels Testing Integration

Uploaded by

Document Information

Original Description:

Original Title

Copyright

Available Formats

Share this document

Share or Embed Document

Sharing Options

Did you find this document useful?

Is this content inappropriate?

Copyright:

Available Formats

Levels Testing Integration

Uploaded by

Copyright:

Available Formats

Different Levels of Testing

• The entire system is viewed as a collection of

• Goal: Test all interfaces between subsystems

• The Integration testing strategy determines the

• To make sure that

• Unit tests only test the unit in isolation

• Many failures result from faults in the interaction of

• Often many Off-the-shelf components are used that

• Without integration testing the system test will be very

• Failures that are not discovered in integration testing will

• stub: A controllable replacement for an existing

• useful for simulating difficult-to-control elements:

• also useful when dealing with brittle legacy code/systems

• Identify the external dependency.

• Extract the core functionality of the object into

• Write a second "stub" class that also implements the

• mock object: A fake object that decides

• A mock waits to be called by

Test G Test D,G

• Test the top layer or the controlling subsystem

Layer I Layer I + II All Layers

• Combines top-down strategy with bottom-up

• Top and Bottom Layer Tests can be done in

• Problem: Does not test the individual

• Solution: Modified sandwich testing strategy

• Requires integrated tool support:

• daily build: Compile working executable on a daily basis

• smoke test: A quick set of tests run on the daily build.

• For Top-down approach with n nodes,

• For Bottom-up approach with n nodes and

• For both cases there may be as much as

• There are 5 nodes and 4 leaves and 4

1. n- 1 = 5 -1 = 4 stubs (Top-down) (Bottom-up)

B Number of test sessions = 6:

You might also like