You are on page 1of 9

Protocol testing techniques

Baoyu Wang* and David Hutchison t provide a survey of testing techniques


for communication protocols

that the protocol is working under actual running


This paper deals with testing techniques for communication conditions. Traditional testbeds for testing protocol
protocols. Several protocol testing support systems implementations are separately designed by suppliers
designed for testing protocol layers of the ISO/OSl and users to their own requirements. Obviously, there are
(International Standards Organisation/Open Systems two main disadvantages. First, the cost of building
Interconnection) Reference Model are introduced. These individual testers will be high. Second, as different
systems are mostly designed for the conformance testing products choose their own reference product for
of protocols produced by third parties, which is intended communication, standardization for protocol products
not only to provide test sen/ices for both suppliers and will hardly be attained. As a result of these, a third party
users but also to meet the requirements of standards. can be employed to test products in order to eliminate
Several test case generation methodologies are described, both disadvantages.
and the authors' experience in using one of the methods is
The second area is to create or to apply a suitable
presented. method for generating efficient test cases to test
implementation products. During one test, thousands of
Keywords: communication networks, protocol testing
test cases may be executed. How can finer test cases be
conformance, third party testing
applied, and yet all the errors surely found? How can a test
system generating test cases require less human involve-
ment? These are the main research interests in this
In data communications it is generally accepted that area.
protocols should be standardized, otherwise it will be
hard to reach the goal of open communication among
different systems. However, users must be assured that CONFORMANCE TESTING
products based on the same standard but produced by
different suppliers can communicate with each other The term 'testing' indicates that some operations are
successfully and reliably. Protocol testing can be used to carried out to certify the conformity of an implementation
show whether the products under test meet the standard to its specification. In data communications, protocol
specification. In this situation suppliers and users may rely testing means that the test object is a protocol implemen-
on independent agencies who provide a test service for tation. It can be simply defined: executing a sequence of
the conformance of protocol products to standards (such coordinated operations to drive the implementation of
an agency is called a third party). the protocol through a given sequence of input stimulations
How are communication protocols tested? The major and observing how the implementation responds to well
effort in testing has been in two areas. The first area determined actions. However, testing a protocol also
involves building a support test system; this is because the involves testing the services which are provided by the
implementation of protocols should be tested by a protocol. Such testing is called 'service testing', and it
trusted independent assessment system. The test system verifies the interaction of the protocol implementation
should provide a simulated environment which ensures with its user. Faults detected in service testing can be
evidence of a problematic protocol implementation, but
*Computer Centre, Nanjing Institute of Technology, Nanjing, Jiangsu, the absence of faults detected in service testing may be
People's Republic of China insufficient to declare that the protocol implementation is
tDepartment of Computing, University of Lancaster, UK correct.
0140-3664/87/020079-09 $03.00 © 1987 Butterworth & Co (Publishers) Ltd
vol 10 no 2 april 1987 79
In software engineering, testing can usually be designed UK's well known NPL assessment centre, will be given
from either a functional or a structural point of view. below.
Structural testing is often called 'white-box' testing which
is particularly used by the implementer to check the NPL assessment centre
internal operation of a program. Functional testing is
commonly known as 'black-box' testing: it treats the The National Physical Laboratory (NPL) in the UK started
program or system as a black-box and assumes that the to develop appropriate testing techniques for protocol
internal structure of the implementation is not available. implementation assessment in 19801, and is considered
For communication protocol testing both sorts of tests one of the pioneers in the field of protocol testing. The
are necessary. It is also necessary to have a trusted main objective of the NPL is to establish techniques
protocol (a 'peer protocol' or reference protocol) to applicable to international standards. Their approach is to
communicate with. A reference protocol implementation test one layer at a time (layer N) by checking its
should be connected with the implementation under test conformance to the ISO/OSI Reference Model, and all
in order to access the test object within an operational layers below the layer under test (layer N-l, N-2, etc.)
environment. If the protocol conforms to its specification, are assumed to have been adequately tested. Early work
the communication should be successful, without any in the field has provided a basis for the development of
error. Normally the tests are done by two parties: one is many protocols, and concentrates on the requirements
the supplier and the other the user. For communication for testing OSI products by using a real network
protocol products, third parties are active in testing connection 2.
product conformance to the ISO's (International Standards The simple physical architecture of the assessment
Organization's) OSI (Open Systems Interconnection) centre consists of three components:
standard network model.
The supplier, as the first party, should test the AT • NET ~ SUT
conformance of a product to its specification and obtain The AT (active tester) communicates with the SUT
enough evidence to show how reliable the product is. (system under test) via a communication medium (NET).
The user, as the second party, does not need to At the N PL, British Telecom's Packet SwitchStream (PSS) is
understand different implementations of a protocol, and used as the primary communication medium. The test
is only concerned with services provided. As the service can be conducted many miles away from the system
specification is semantic-free, all correct implementations under test. When the (N-l) service is not end-to-end, an
of a protocol should provide the same services. The users advanced physical architecture will be used. This
have to ensure that the products can meet their own architecture inserts a transportable box called the EMU
requirements and thus will set certain criteria to measure (environment manipulation and monitoring unit) between
whether the product is of good quality. The third party is the communications medium and the client's system.
independent from both suppliers and users, and normally The AT is the key part of the assessment centre and has
provides several testers which are designed for individual been developed in two phases: the first adopts the simple
layers of the OSI standard. Any OSI products to be tested physical architecture and is restricted to the types of errors
are connected with the test system so that conformance that the SUT can be subjected to. The second uses the
testing can be carried out in a standard communication advanced physical architecture which includes the test
environment. driver, communication medium and EMU, and is closer to
During the last few years, interest in third party testing the real structure of the corresponding OSI service and
of communication protocols has been gradually increased protocols.
internationally. At present most test systems deal with the The logical architecture of assessment testing can be
network and transport protocols of the OSI standard. described in terms of the OSI layers as indicated in Figure 1.
Work is continuing on test facilities for the higher layers. A
successful tester should include an efficient test case -'=--- Active tester Client's system
generator, therefore researchers are investigating method-
ologies for generating more effective test cases, in order to [ Test driver [= Test driver _I I
Responder protocol - I Test responder
achieve the goal of automatic testing protocols with very
little or no human interference.
I
(N) protocol+err J_
encoder/decoder r
(N)- protocol
Errors
_J
-J
I
Implementotion
under test (TUT)
SURVEY OF TESTING S Y S T E M S

In recent years, several protocol testing systems have


I_ (N-I) protocol I
been designed world-wide, specifically in the UK, USA, (N- I ) protocol I-- =I (N- I ) protocol
Canada, France, Italy and FRG. Most of these have already implementotion ] I implementation
exception
been put into operation, and all test systems are built for generator
particular layers of the OSI Reference Model. Underlying communicotion channel
A brief introduction to test systems and tools, as well as
comparing their major characteristics with those of the Figure 1. Logical structure of assessment centre

80 computer communications
In its initial model, this testing system was built for
testing protocols for the network layer of the OSI model.
J Datosink J J Scenariofile I
Later on, however, it proved to be satisfactory for testing
protocols above the network layer when the underlying
service was expected to be end-to-end. The NPL is I o g ~ I Operator
COnSOle
working on adapting the active tester from layer-by-layer /
testing to multilayer testing, which will not consider
intermediate interfaces, so that the testing process will Test centre
become more manageable. Now NPL is takingthe lead in driver(TC) Datagenerator
developing an OSI standard for an OSI conformance
~ 1
testing methodology and framework.
Exception
generator T r
o nsport
protocol
implementation
NCC service
At the National Computing Centre (NCC) in the UK, a
_ I Network I
third party testing service has been built under the name --i interface I
of NCC Comms-AID (Communication Software Assess-
ment and Interactive Development) 3. 1
Networkprotocol
The NCC objectives are2:
Figure 2. NBS tester
• to establish an effective resource to assist the OSI user
community in the UK;
• to provide test facilities for as many protocols as valid events at the service interface or from the peer
practicable; protocol; that it can reject errors passed across the service
• to provide a better indication of commercial feasibility; interface and peer protocol errors, and that it can handle
• to test and develop further techniques; timeouts or multiplexed connections correctly as well 4.
• and to promote the OSI model and the concept of Although this system has been successfully used in
third party testing. transport layer protocol testing, the NBS considers that it
is unreasonable to construct one testing system for each
The NPL and NBS test systems (see below), which are
protocol. Consequently, an adaptation of this system has
separately used in testing network and transport layer
been planned for use in testing other layer protocols of
protocols were both installed at the NCC in 1983. The the OSI model.
NCC Com ms-AID service is a remote testing facility which
includes a tester located at the NCC site, the product
under test is installed on the client's own system, and the Two Canadian testing systems
connection of both sites is made via PSS. The tester can
generate test data and interpret responses as well. The Bochmann's group from the University of Montreal have
significant feature of this system is that the test can be developed some systematic methods to generate test
controlled either at the NCC or at the client's site. sequences for communication protocols, particularly for
NCC test facilities are already available for testing the transport layer protocols s. They have designed a simple
network layer and most parts of the transport layer. Higher and general test architecture as an experimental bed (see
layer testing facilities are under development. The NCC Figure 3) to try their methods for generating test cases. The
will use their system to provide a commercial testing test bed consists of an active tester, connected to a test
service. responder.
At the BNR laboratory in Ottawa, an integrated test
centre has been built for testing SL-10 (a trade mark of
NBS testing system Northern Telecom) packet network products. This centre
provides four test tools which make the network testing
The National Bureau of Standards (N BS) in the USA started more manageable for the implementer, manufacturer
to build a certification centre in 19824. The initial purpose and network operator, and also automatically allows
was to meet the requirements of testing their own
protocol implementation conformance to Federal
Information Processing Standards. Activetester L_ Testprotocol - [ Testresponder
The architecture of the tester was designed for the
transport layer of the OSI standard as shown in Figure 2. Peerunit I !I Unit undertest
Both tester and client are connected via the network
protocol during testing. The major difference from NPL's Network protocol I-- --j Network protocol

tester is that it does not require the test driver-responder


protocol. I Communication link
I
The aims of this testing centre are to ensure that the
implementation under test can respond correctly to all Figure 3. A test bed structure

vol 10 no 2 april 1987 81


terminals and host computers as well as total networks to Certificotion
centre (STQ) Vendor test
be tested for conformance and performance 6.
The function of these four tools can be simply I
Test driver ~.~ J Test protocol Test responder
described as follows: IPT (Interactive Protocol Tester) is
for checking protocol implementation against specification;
NLTS (Network Load Test System) generates simulated
'
N reference I N protocol N entity under
test I
entitt = I (N-I) service
traffic and measures the network's performance; NPM
(Network Process Monitor) examines operating software Public network
at the module level; and NTS (Network Test System)
provides an automatic or interactive test sequence for Figure 4. STQ test system
coordinating and controlling distributed network testing.
Each tool can be operated individually by using a TC SUT
standard data terminal. IPT, NLTS and NPM tools run on
SL-10 network processors and can be operated remotely
via the network or the NTS Driver. By using these tools in
Ts 1
scenorio (N-I) service
AR
(N)EUT
combination, a fully integrated and automatic operational
test centre can be achieved. CT
CUT

Figure 5. STP test system; TC = test centre, TS = testing


French test systems system, SUT = system under test, AR = astride responder,
(N) EUT = N entity under test, CT = connection for
France has been the most prolific source of development testing, CUT = connection under test
of testing systems 2. Before 1983, there was the ADI group
which worked on the RHIN project. This project developed testing system (TS) is different from most other test
three test systems7 for protocol implementation, namely systems (but see also the NBS system in Figure 2) because
CERBERE, GENEPI, and STQ. Since 1984 another group it employs a 'scenario' which basically generates test
called BULL has been working on building an OSI cases, rather than by using a reference implementation.
protocol testing environment based on a distributed PDUs are produced by the tester, and the (N-l) service is
testing architecture (a remote testing system and a related used directly to exchange the (N) PDU with (N) EUT.
responder system) called STP8-11. Another difference is that the (N) EUT allows messages
CERBERE is a tool designed to be introduced between from the TS to gain access to either the upper interface
two pieces of equipment (implementation under test and through the astride responder (AR) or the lower interface.
reference implementation) running high level protocols. On the contrary, most other testing systems allow the
It acts as a relay in Layer 3 of the OSl model, and is a message from the active tester to enter only one
portable tester that is connected between the subnetwork interface.
and the system under test. Concurrently, it is able to The STP system has been implemented and is used for
monitor, analyse or perturb the traffic between that interactive testing of a transport entity.
system and the tester.
GENEPI is a protocol data unit generator which was
initially used for testing Layers 4 and 5 of the OSI model, CREI system
and it can now be configured to test a single layer or a pair
of adjacent layers. It takes the same position in the The CREI group in Italy started their research project to
architecture as the encoder/decoder. design a testing system in 198312. The system is intended
STQ (Test and Qualification System) consists of to be used to test protocol implementations in terms of
software designed to run on a certification centre for both protocol and service testing. The basic architecture
testing an (N)-layer protocol. of the system is as shown in Figure 6.
The main characteristic of STQ is that it allows testing of
Layers 4, 5 and 6 of the OSl model separately or
simultaneously, and also it assumes that there is testing Test driver ~ Testingprotocol .~[ Test responder
access directly to the lower layer interface of the (TD) (TR)
implementation under test. The architecture is shown in
Figure 4 and the system is available for testing Layer 4 of
the OSI model. __I I
Finally, STP (System for Testing Protocols) is a system in implementation under test
which the defined responder is simple, small and easy to (PrR) under test (IUT)
implement. This system allows an efficient methodology
for testing OSl protocol entities. The general architecture 1 L
of STP is shown in Figure 5.
The TS of STP system constructs (N)-protocol data F Underlying Ioyer
]
units to be sent to the (N) EUT (entity under test). The Figure 6. CREI test system

82 computer communications
A special feature of this system is that its user can be the test is located at the responder site and a reference
implementer of the object under test, or the operator of product is set at the active tester. All test systems should
the system without making changes in the system have a mechanism for generating test cases so that the
structure. The implementer using the system can work operator does not need to input the test data manually.
locally to debug the implementation under test on line Most systems provided by third parties are designed to
and to perform the test with a remote reference operate on the active tester's local site and to look on test
implementation (PIR). When the test centre operator uses responders as customers. Only a few, like GMD and CREI,
the system, a remote customer implementation (IUT) is allow the test responder to call the tester and can be
tested against the local reference implementation (PIR). operated at both sites.
This system allows the TD (tester driver) to transfer Current standardization work by ISO and CCITT is
information about the service primitives to the TR (tester contributing in the following areas: first, the development
responder) before the testing session starts. Therefore the of a portable test system for the OSI File Transfer Access
TR is required to have the memory and an interpreter for and Management (FTAM) (Layer 7) protocol; second,
storing and performing the TD instructions. automatic generation of test cases from formal protocol
The major functions performed by the TD and TR are: specifications; and third, the development of standards
notification of the testing session opening and closing; for conformance testing and formal protocol description
data needed for the synchronization of the TD and TR techniques.
behaviour; and indication of any failure 12. When building a test system, designing the test case
The TD consists of five modules, namely the database, generator is a vitally important part of the work. Therefore,
operator-interface, control, test processing, and mapping research to find better methods for generating test cases is
module. The TR consists only of the test processing and very valuable.
mapping modules. All modules implemented depend on
the testing layer.
It is proposed that the system should be able to test
any layer of the OSI architecture, or a sublayer, or even the M E T H O D O L O G I E S IN PROTOCOL TESTING
combination of several layers of the OSI.
How can a system be produced with effective test cases
and how can these be automatically generated without
G M D system looking at the details of the implementation? These
problems have attracted researchers to concentrate their
At GMD of FRG, a special protocol tester for testing and attention on test case design methodologies. As a result of
diagnosis of higher level protocols has been developed 13. this, a number of techniques for generating test cases
The structure of the tester is similar to Figure 4 (the French have recently become available. For instance, Sarikaya
STQ system). and Bochmann have found that various methods for
The proposed testing techniques assume that tests are testing state machines can be applied to the selection of
driven under remote control by the client. At the test sequences for protocols specified as a finite state
beginning, the strategy of manually driven tests was used, machine s. They have applied three methods: transition
and included the following: no test driver responder tour, checking sequence and W-method, in testing a
protocol; no constraints for test responders; and a test transport layer protocol. Ural and Probert TM have also
driver manually controlled by the client by using test adapted the method of context-free grammars to protocol
commands. Later, the strategy of automatic tests was testing. Most recently, Sabnani and Dahbura 15 have
developed, based on common agreement of test protocols, created a new method to produce unique input-output
definition of test responders and an active test driver sequences for protocol testing.
which reads the test commands from a file.
AT GMD, the active tester can be accessed via X.25 or
X.28 and can be called as a test partner from the client's
system. It can also be inserted into the connection Transition tours
between two partners for the purpose of arbitration
testing (which means to determine the cause of a The method of transition tours, like the others, assumes
problem) and allows the testing of several parallel test that the protocol to be tested is specified as a finite state
connections. machine (FSM). This is one of the simplest methods.
The GMD project is one of the collaborative projects Transition tour is the name of an input sequence of the
funded by the CEC (Commission of the European FSM, which starts with the initial state and will cover all the
Communities). transitions in the FSM state table at least once. It detects
all operation errors (errors in the output function), but it
does not detect all the transfer errors (errors in the next
Remarks state function).
As the test sequence is based on the traversal of the
As shown above, test systems are commonly constructed FSM graph, the test should start from the initial state, go
by using an active tester connected to a test responder for through all possible transitions, and then come back to
testing OSl or similar products. A protocol product under the initial state.

vol 10 no 2 april 1987 83


b,d

b c

Figure Z A finite state machine example; numbers


represent states, letters represent inputs

In Reference 5 the transport protocol has been used to


illustrate this method. Here a simple finite state machine Figure 8. Testing tree:
(see Figure 7) will be used to show the transition tour P= { { } , b , a , d , a . {b,c},a.c. {d,a,c,b,g} }
sequence. PW = {b,b.b,a.b,d.b,a. {b,c}.b,a.c. {d,a,c,b,g}.b}
Assuming that State I is the initial state, a transition tour
sequence may be: The W-set is constructed using aspecial method 17, and
the P-set can be formed from atestingtree ~8,which shows
Input a b b d a c d c a c b a c g
State 1 2 1 1 1 2 3 3 1 2 3 1 2 3 1
every transition from State i to State j on each input.
However, the limitation of using this method is that it is
not certain that every finite state machine will have a W-
The sequence starts from and terminates at State I and
set sequence, especially if it is an incompletely specified
will direct the machine to go through all possible
machine. Therefore, if intending to use this method, one
paths. should first make sure that the defined machine has a W-
This method was first demonstrated by Naito 16 in his
set sequence. When a machine does not have a W-set, a
paper on software testing. Later, Sarikaya and Bochmann
procedure can take place to form a completely specified
applied it in testing a transport layer protocol implemen-
machine which has a W-set, for example, adding an 'error'
tation, and found that the method was applicable to
state and declaring all unspecified transitions to lead to
incompletely specified machines if the machines were
this state.
strongly connected, contrary to the original assumption
that the machine under test should be minimal, strongly
connected and fully specified. This method is generally
applicable to all protocols specified by a finite state Checking sequences
machine. The limitation is that the fault detection
capability finds operation errors but not transfer errors. A checking sequence is a test sequence which consists of
three parts:
• Initial sequence
W-method • State recognition sequence
• Transition checking sequence
The W-method involves two sets of input sequences: one
is the W-set, the other is the P-set. The W-set is a Users of this method should ensure that the FSM under
characteristic set of the minimal FSM, and consists of test will have a 'distinguishing sequence' (DS). The DS is
input sequences that can distinguish between the similar to the W-set mentioned above. It is an input
behaviours of every pair of states. The P-set consists of all sequence, on which, for each initial state, the machine
partial paths. will produce a different output sequence. One can get a
The W-method gives a set of test sequences formed by DS by constructing a successor tree 19 for the FSM.
the concatenation of the W-set and P-set. Each test At the start of testing, one should first use an initial
sequence starts with the initial state and returns to it again sequence to bring the FSM to the initial state. Then a state
afterwards. It is also guaranteed to detect any misbehaviour recognition sequence is used to show the response of
of the machine. each state to the DS, followed by a transition checking
We also use the example in Figure 7 to describe how to sequence to check all individual transitions in the state
form a P.W set. As the W-set sequence should distinguish machine.
the behaviours of every two states, input 'b' should be the Each transition from a certain state using input Xi (say)
W-set for this finite state machine. The P-set can be is checked by applying the sequence Xi.DS. The checking
obtained from the tree in Figure 8. sequence is formed by the concatenation of all Xi.DS

84 computer communications
sequences. The problem is that when the number of • Compute all input/output sequences of length 1 for
states is large it is difficult to find a DS in a specified each state;
machine. Normally, a 'read state' input/output facility is • Check if these are unique; if yes, a UIO sequence for
added to help obtain a DS. the state has been found;
• If not, compute sequences of length 2 for the state
which does not have a UIO sequence so far;
Context-free grammars • Continue to compute length I + 1, and check unique-
ness until the UIO for every state is found.
Grammars are often used to define languages. The
The second step is to generate a test sequence, as
method of context-free grammar which will be discussed
follows:
here uses an attributed grammar to build a program
language for specifying and generating test sequences (1) Generate a test to check whether each state is
from the service or protocol specifications. reachable from the initial state and whether it has a
The use of an attributed grammar to generate test cases UIO sequence (by using the method above);
in software testing was suggested by Duncan and (2) Generate a test to check whether each edge has the
Hutchison 2°. Ural and Probert have adapted this method correct head state, the correct tail state, and the
from its use in general software testing, and have correct label;
developed a method to generate test sequences for (3) Concatenate the tests from (1) and (2), and remove
transport layer service and protocol testing TM. When the redundancy in order to reduce the test sequence
protocol is specified by using a context-free grammar, it length.
can easily be tested using this method.
When using this method in testing protocols, it is The format of the test sequences is the edge label
sequence:ll/01 12/02 13/03 . . . ; the label of an edge
necessary, first, to understand correctly the functional
requirements of the communication protocol; second, an contains the input part (I) and expected output part (O).
attributed context-free grammar is constructed and test The test sequence is able to give a clear overview of the
sequences specified from a functional requirement; third, input and output behaviour, and it is therefore very
a language defined by the grammar is used to specify a helpful to carry out a result comparison with it. When one
generator for test sequences; fourth, the grammar is takes the input part of the labels as a test input sequence
implemented as a generator program; finally, the programs to do the protocol testing, a sequence of results is
are run in a controlled fashion and a set of representative obtained. If there is any difference between the actual
test sequences generated randomly, systematically, or result and the output part of the test sequence, it is
selectively. obvious that the protocol under test is faulty.
Although using the test sequence generator can This method has also been applied to the Alternating
Bit Protocol 15 and the Proway protocol 22.
automatically produce test sequences, the generator
design needs careful human insight. Therefore, this is
called a semiautomatic method.
Comparison

UlO sequence generation Except for the context-free grammar method which is
mainly used when the protocol is specified by a context-
Sabnani and Dahbura is have introduced another tech- free grammar, all others are based on the finite state
nique for generating protocol tests by means of a novel machine specification. Bochmann has carried out a
procedure which can generate test sequences. The comparison on the first three methods mentioned above.
procedure is based on the assumption that the protocol is For fault detection capability, the checking sequence is
specified as a minimal finite state machine. The key idea best, and the W-method is better than the transition tour.
in the procedure is first to compute a unique input- For the length of sequence, the transition tour gives the
output (UIO) sequence for each state in the protocol shortest test sequence because the test sequence is
specification, and then to generate a test sequence to visit directly formed by checking the FSM diagram, and does
all states and state transitions by using the UIO sequences. not need to add any extra characteristic sequence. For the
The testing covers all the state transitions and checks the W-method and checking sequence methods, the problem
UIO sequences for each state. It can also detect the is that sometimes it is difficult to find the DS or W-set for a
problems that exist in the implementation, such as an finite state machine module, resulting in much more work.
incorrect edge label, an incorrect header state or an All these methods test only whether every path is
incorrect tail state. reachable or not, and do not refer to further test cases
The procedure for generating test sequences consists chosen.
of two main steps: The UIO sequence technique is an advance on the
First compute a UIO sequence for each state. Computing others in that it is an approach to generating tests
the UIO sequence includes: automatically. The technique emphasises testing of each
state rather than testing the whole machine. Any level of
• For each edge label, compute the list of edges with that protocol which is specified by a FSM can be tested by this
label (for later use): technique. Those who do the testing need only input an

vol 10 no 2 april 1987 85


edge set which consists of head state, tail state and edge presented and some methods used in generating test
label, together with the num ber of edges, and then a serial cases described.
test sequence will be generated for each state and each Existing support test systems are all designed for
edge. A great advantage of this technique is that it is particular layers of the OSI model, especially for testing
possible to make short test sequences because although network and transport layer protocols. These systems are
each state has a different UIO sequence, the majority of successfully used in fulfilling their original g o a l -
UIO sequence lengths are one. conformance testing for protocol standardization. With
such testing systems, standard protocol products will
have a large market and data communication between
EXPERIENCE IN IMPLEMENTING A UlO different devices will be much easier than before.
GENERATOR Finding a new and simple method for generating test
cases is very important in the protocol testing area. As the
As has already been described, the technique for specification of a protocol is usually described formally in
generating UIO test sequences is fairly straightforward a standards document, designing a test case generator will
and the ideas involved are easy to understand. The range not generally be too difficult. In this paper, the five
of usage is so wide that any protocol specified by a finite methods introduced can all be used to generate test cases
state machine can use it directly to generate the test for communication protocols, and most of these aim at
sequences unless it is not a minimal FSM. Nevertheless, it automatic testing. Once the active testers are armed with
is enough to add only a minimization process to those automatic test case generators, automatic testing can be
exceptions. The authors have implemented this technique 22 fully realized.
and found that the overall test sequence generator
consists of four basic parts:
• The UIO sequence generator: generates a unique input ACKNOWLEDGEMENTS
and output sequence for each state.
• The shortest path generator: generates a sequence to The authors are grateful to the two referees for their
find the shortest path for each state. helpful and detailed criticism of the original version of this
• Edge test sequence generator: generates a test paper.
sequence for each edge, the sequence being formed
by the shortest path of the edge head state, plus the
edge label, plus the UIO sequence of the edge tail REFERENCES
state.
• State test sequence generator, generates a test sequence 1 Rayner, D 'A system for testing protocol implemen-
for each state, and is formed by the shortest path of the tations' in Sunshine, C (ed) Protocol specification
state, plus the UIO sequence of this state. testing and verification North-Holland (1982)
The unoptimized test sequence is the concatenation of 2 Davidson, I 'Testing conformance to OSI standards'
two state and edge test sequences. In order to remove Comput. Commun. Vol 8 No 4 (August 1985)
sequences that are completely contained in others and pp 170-1 79
thus make the test sequence shorter, another comparison 3 Davidson, I 'Independent testing of protocols' Proc.
procedure is needed. This is a fifth part of the test Networks '84 (June 1984)
sequence generator. 4 Nightingale, J S 'Protocol testing using a reference
The authors have applied the technique to generate implementation' in Sunshine, C (ed) Protocol speci-
test sequences for testing Proway21 protocol implemen- fication testing and verification North-Holland (1982)
tations. The inputs were entirely based on the state machine 5 Sarikaya, B and Bochmann, G V 'Some experience
diagram of the protocol specification. After processing the with test sequence generation for protocols' in
inputs, the generator produces a set of test sequences Sunshine, C (ed) Protocol specification testing and
automatically; these sequences are directly input to the verification North-Holland (1982)
implementation under test. Therefore, test results were 6 Hornbeek, M W A 'An integrated test centre for SL-10
easily obtained without doing any more work. When the packet networks' ACM CompuL Commun. Rev. Vol
total results including the test sequences generated by 15 No 4 (September 1985)
the procedure were obtained it was necessary to only 7 Ansart, J P 'A protocol independent system for
compare the outputs produced by the implementation testing protocol implementation' in Sunshine, C (ed)
with the output part of the test sequence. Protocol specification testing and verification
The authors' experience of implementing and using North-Holland (1982)
this technique shows that it is simple and effective and 8 Rafiq, O and Haddad, J 'Description of protocol
applicable for testing most protocols. testing scenarios' COMNET '85 (Conference on
'Services conveyed by computer networks' Budapest,
Hungary (1-4 October 1985)
CONCLUSION 9 Rafiq, O 'Tools and methodology for testing OSI
protocol entities' Int. Symp. Fault Tolerant Computing
In this paper a survey of protocol testing systems has been IEEE, Ann Arbor, USA (19-21 June 1985)

86 computer communications
- 0 - 0 0 0 0

10 Rafiq, O 'A good approach for testing protocol Rev. Vol 15 No 4 (September 1985)
implementations' Int Semin. CompuL Network. Perf. 16 Naito, S and Tsunoyama, M 'Fault detection for
EvaL Tokyo, Japan (18-20 September 1985) sequential machines by transition tours' Proc. IEEE
11 Rafiq, O et al. 'Towards an environment for test OSI Fault Tolerant Computing Conf. (1982)
protocol' Fifth International Workshop on Protocol 17 Gill, A Introduction to the theory of finite state
Specification, Testing and Verification Toulouse- machine McGraw Hill, USA (1962)
Moissac, France (10-13 June 1985) 18 Chow, T S 'Testing Software Design Modelled by
12 Palazzo, Set al. 'A layer independent architecture for Finite State Machines' IEEE Trans. on Software
a testing system of Protocol Implementations' in Engineering Vol SE-4 No 3 (May 1978)
Rudin, H and West, C H (eds) Protocol specification 19 Kohavi, Z Switching and finite automata theory
testing and verification Elsevier Science (1983) McGraw Hill, USA (1978)
13 Giebler, A 'Testing and diagnosis aids for higher level 20 Duncan, A G and Hutchison, J S 'Using attributed
protocols' in Rudin, H and West, C H (eds) Protocol grammars to test designs and implementations' Proc.
specification testing and verification Elsevier (1983) 4th Int. Conf. Software Engineering (1981)
14 Ural, H and Probert, R 'User-Guided Test Sequence 21 British Standards Instilution Draft--Process data
Generation' in Rudin, H and West, C H (eds) highway, type C, for distributed process control
Protocol specification testing and verification Elsevier system (1985)
(1983) 22 Wang, B 'Implementation and testing of computer
15 Sabnani, K and Dahbura, A 'A new technique for network protocols' M. Phil Thesis, University of
generating protocol tests' ACM Comput. Commun. Lancaster, UK (1986)

vol 10 no 2 april 1987 87