Innovations Syst Softw Eng (2008) 4:71–85 DOI 10.

1007/s11334-007-0042-z

ORIGINAL PAPER

Patterns: from system design to software testing
Neelam Soundarajan · Jason O. Hallstrom · Guoqiang Shu · Adem Delibas

Received: 16 December 2007 / Accepted: 19 December 2007 / Published online: 12 January 2008 © Springer-Verlag London Limited 2008

Abstract Design patterns are used extensively in the design of software systems. Patterns codify effective solutions for recurring design problems and allow software engineers to reuse these solutions, tailoring them appropriately to their particular applications, rather than reinventing them from scratch. In this paper, we consider the following question: How can system designers and implementers test whether their systems, as implemented, are faithful to the requirements of the patterns used in their design? A key consideration underlying our work is that the testing approach should enable us, in testing whether a particular pattern P has been correctly implemented in different systems designed using P, to reuse the common parts of this effort rather than having to do it from scratch for each system. Thus in the approach we present, corresponding to each pattern P, there is a set of pattern test case templates (PTCTs). A PTCT codifies a reusable test case structure designed to identify defects associated with applications of P in all systems designed using P. Next we present a process using which, given a system designed using P, the system tester can generate a test suite from the PTCTs for P that can be used to test the particular system for bugs in the implementation of P in that system. This allows the tester to tailor the PTCTs for P to the needs
N. Soundarajan (B) · G. Shu · A. Delibas Computer Science and Engineering, Ohio State University, Columbus, OH 43210, USA e-mail: neelam@cse.ohio-state.edu G. Shu e-mail: shug@cse.ohio-state.edu A. Delibas e-mail: delibas@cse.ohio-state.edu J. O. Hallstrom School of Computing, Clemson University, Clemson, SC 29634, USA e-mail: jasonoh@cs.clemson.edu

of the particular system by specifying a set of specialization rules that are designed to reflect the scenarios in which the defects codified in this set of PTCTs are likely to manifest themselves in the particular system. We illustrate the approach using the Observer pattern. Keywords Design patterns · Contracts · Testing

1 Introduction Design patterns [4,11,24] have had a profound impact on how software systems are built. This is not surprising since patterns capture the distilled wisdom of the software community, and provide effective solutions to recurring design problems. They allow software engineers to reuse these solutions, tailoring them to the needs of their particular systems, rather than reinventing them from scratch. Patterns are used primarily during the design phase of software systems. In this paper, we focus on later stages of the software life-cycle, the testing and maintenance phases. Specifically, the question we are interested in is, how can system designers and implementers test whether their systems, as implemented, are faithful to the requirements of the underlying patterns? This question is especially important for large systems since such systems tend to have correspondingly large software teams and hence a correspondingly greater likelihood that different members of the team may have subtly different interpretations of the patterns underlying the system’s design. Hence the team members may implement their respective parts of the system in ways such that when these parts are put together, the resulting overall system is not compatible with the patterns’ intent. Design-related bugs of this kind may not manifest themselves in test cases that are developed using standard approaches such as testing against the functional requirements or even code-based criteria such as statement,

123

72

N. Soundarajan et al.

or branch, or path coverage [3]. The testing techniques and supporting tools discussed in this paper will be of value in reducing the likelihood of such bugs going undetected, and will aid in their localization. The problem of ensuring that a system is faithful to the patterns underlying its design is even more serious during the maintenance phase since system maintenance may be the responsibility of team members who were not involved with the original design or implementation of the system— and consequently have only a superficial understanding of its design. As a result, over time, the design integrity of the system is likely to erode. The approach we develop in this paper can help address this aspect of the problem as well. This is because the test suite used to test whether the system is faithful to its underlying patterns can be included as a key part of the system’s design documentation in the same manner as standard unit tests or other functionality tests. As the system evolves during maintenance, the design test suite will help system maintainers identify and localize changes that may conflict with the original design. If one of these tests were to fail, that doesn’t necessarily mean that the changes in question are flawed. It may be that the design needs to be modified; but this decision should be made consciously with the design documentation being appropriately updated, rather than simply modifying the code. We will return to this point in the final section of the paper. In standard software testing, we test software against its specifications. These specifications must be formal since, otherwise, there is no way to conclude whether a test was successful or not1 . Similarly, to test whether a system is faithful to the intent of the patterns underlying its design, we need formal specifications of the patterns. In previous work [12,26,31], we presented an approach to providing precise specifications for patterns and their applications in the form of pattern contracts and subcontracts. It is against these contracts and subcontracts that we wish to test individual systems to validate their design correctness. An important aspect of this approach to pattern specification is that the requirements and behavioral guarantees specified in a pattern contract apply to all uses of the pattern; a subcontract characterizes how the pattern is specialized in a particular application. Thus the wisdom of the community captured in a design pattern is reflected in the corresponding pattern contract; it can be reused by every team that uses the pattern in the design of its system. The individual team need

1

When is a test successful? One definition in the testing literature [3] says that a test is successful when it shows that the software does not meet its specification, since the purpose of the test is to find flaws in the software. Others [20] use the convention that a test succeeds, or that the software “passes” the test if, during the test execution, the software behaves according to its specification. We will use this latter convention in this paper.

only create the subcontract that specifies how the pattern is specialized for use in a particular application. A similar consideration is a primary motivation for the approach we present in this paper. That is, a key goal of our work is that the testing approach should enable us, in testing whether a particular pattern P has been correctly implemented in different systems designed using P, to reuse the common parts of this effort rather than having to implement it from scratch for each of these systems. Thus in the approach we develop, corresponding to each pattern P, there is a set of what we will call pattern test case templates (PTCTs). A PTCT codifies a reusable test case structure designed to identify defects associated with applications of P in all systems designed using P. Next we present a process using which, given any system designed using P, the system tester can generate a test suite from the PTCTs for P that can be used to test the particular system for bugs in the implementation of P. The process allows the system tester to tailor the set of PTCTs corresponding to P to the needs of the particular system by specifying a set of specialization rules that are designed to reflect the structure and the scenarios in which the defects codified in the set of PTCTs for P are likely to manifest themselves. An example will help illustrate our approach. Consider the classic Observer pattern which we will use as our primary case study throughout the paper. According to the standard description [4,11] of the pattern, participating objects play one of two roles: Subject or Observer. The intent of the pattern is to keep multiple observers consistent with the state of a single subject. When an object wishes to become an observer of a subject, it invokes the subject’s Attach() method. When it is no longer interested, it invokes the Detach() method. In addition, Subject includes a Notify() method which, according to the intended usage of the pattern, is required to be called whenever the state of the subject is modified. Notify() is required to invoke Update() on each attached observer which must, in turn, update the state of that observer to make it consistent2 with the current subject state. The standard UML diagram for the pattern appears in Fig. 1. The subject maintains a set of references to the currently attached observers in the variable _observers. According to the standard description, when Attach() is invoked, it adds a reference to the attaching observer to _observers. But if this was all Attach() did, the intent of the pattern would be violated; this is because the newly attached observer’s state may not be consistent with the current state of the subject – it will remain inconsistent until the next
2

What precisely terms such as “modified” and “consistent” mean is not clear from such informal descriptions. The formal contract for Observer which we will see in the next section will resolve this and other ambiguities in the informal description.

123

Patterns: from system design to software testing
Observer

73

Subject _observers +Attach(in Observer) +Notify() +Detach(in Observer) 1 *

+Update()

for all o in _observers o.Update()

ConcreteSubject −subjectState 1 *

ConcreteObserver −observerState +Update()

Paper organization. Section 2 summarizes our approach to pattern contracts and their relation to PTCTs. Section 3 develops the essential ideas of PTCTs and how they are defined based on pattern contracts. A number of PTCTs for Observer are presented based on the contract for the pattern. Section 4 considers how test cases, for a particular system designed using a given pattern, may be generated from the PTCTs for the pattern; a simple system, the simulation of a Hospital, is used for illustration. Issues of coverage (and the associated metrics) in the context of pattern-centric testing are also discussed. Section 5 briefly surveys elements of related work. Section 6 concludes with a summary of the approach, and its relation to test-driven design (TDD).

Fig. 1 Observer pattern

2 Pattern contracts The contract for a design pattern P consists of a set of rolecontracts, one corresponding to each role of P, and a portion that corresponds to the pattern as a whole. The role-contract for a role R lists the state components of R, and specifies, in standard pre-/post-condition format, requirements that must be satisfied by the various methods of R. In a system designed using P, a class C playing the role R will typically provide other methods in addition to those “named” in R. If these “other” methods were to behave arbitrarily, then the intent of the pattern would be violated even if the methods corresponding to the named methods behaved according to their specifications in the role-contract. In order to eliminate this possibility, R’s role-contract will also include an others specification that must be satisfied by all methods of C, except those corresponding to the ones named in the role-contract. The portion of the contract that corresponds to the pattern as a whole consists of an invariant over the (role) states of the various objects enrolled, at runtime, in an instance of the pattern. The invariant will be satisfied whenever control is not inside any of the methods of any of the participating objects. One potential problem with formalizing a design pattern is that its flexibility might be reduced or even eliminated. A number of features of our formalism help guard against this, the most important of these being the notion of an auxiliary concept. An auxiliary concept is a relation over one or more states of one or more objects interacting with each other according to the pattern. Auxiliary concepts are used in the role-contracts and the pattern invariant, but their definition are not part of the pattern contract. Instead, definitions tailored to particular systems, are provided in the subcontracts corresponding to systems. The subcontract for a particular system also includes a set of role-maps, one corresponding to each class C playing a role R in the pattern, as applied in this system. The C-R-role-map specifies how the state, i.e., variables, of C map to the state of R (listed in its

notification cycle. To address this problem, Attach() must invoke Update() on the newly attached observer. Failure to do this is a common bug that appears in systems designed using the Observer pattern. It is also a bug that can be missed during standard (functional) testing since the inconsistency of the newly attached observer may last only for a short time, depending on when exactly the next modification in the subject state takes place, resulting in a notification cycle which will remove the inconsistency. In our approach, given that this problem is common to systems designed using the Observer pattern, one of the PTCTs for this pattern, as we will see, will correspond to this potential bug. When testing a system designed using Observer, the system tester will specialize this PTCT appropriately to generate a set of test cases that will help test the particular system for the presence of this bug. These tests will confirm that when a new observer in this system attaches to the subject, it is appropriately updated. In other words, these tests will help ensure that the system is faithful to (this particular aspect of) the intent of the Observer pattern; and if the system were to be modified during system evolution, the test cases and the associated documentation will help the system maintainers test for this bug and to identify any violations of the intent of the underlying pattern that may be inserted during this phase. Thus, the approach will help preserve the design integrity of the system as it evolves. Although some approaches to formalizing patterns have been proposed [8,10,19,23] (in addition to our work cited above), we are not aware of any other work focused on the question of testing a system to see whether it correctly implements the patterns underlying its design. Given the widely recognized importance of design patterns and their use in designing and implementing large software systems, we believe that this is an important concern and believe that the approach we present will prove valuable for this task.

123

74

N. Soundarajan et al.

role-contract), which methods of C correspond to each of the named methods of R, etc. As noted in Sect. 1, the pattern contract applies to all systems designed using the pattern; the subcontract specifies how the pattern is specialized for use in a given system. Correspondingly, the pattern test case templates that we will develop in the next section and will apply to all systems designed using the pattern, will be based on the pattern’s contract. The test cases for a given system will be obtained by specializing the templates using the information in the subcontract and accounting for additional application-dependent factors, as we will see later. Consider the Observer pattern. The intent of the pattern, as noted earlier, is to ensure that when the state of the subject is modified, the observers “attached” to the subject are appropriately updated so that their states become consistent with the current state of the subject. “Modified” does not, however, mean a change in any arbitrary bit or byte of the subject state; rather, some modifications are important enough to require updates of the attached observers and others are not. At the same time, which modifications are important enough in this sense and which are not varies from system to system. Hence, we use an auxiliary concept, Modified(), a relation over two states of the subject, to distinguish “important” changes in the subject state from “unimportant” ones. Similarly, what it means for an observer state to be “consistent” with the subject state varies from system to system; and, indeed, from one kind of observer that may be attached to the subject to another that may be attached to the same subject. Hence we use a second auxiliary concept, Consistent(), a relation over an observer state and a subject state, to represent this notion. The contract for Observer, a portion of which appears in Fig. 2, is written using these concepts and specifies the requirements that apply to all systems designed using this pattern. Many patterns are concerned with the sequences of interactions between various objects, i.e., the sequences of method calls invoked on the various objects involved in the pattern. Thus, as we noted in Sect. 1, the Notify() method of the Subject role is required to call Update() on each of the observers currently attached to the subject. Such requirements are expressed in the pattern contract using the trace τ , a ghost variable [15] provided by the formalism. In effect, when a method m() starts execution, a corresponding trace τ , initialized to the empty sequence, is automatically created. Each call that m() makes to a named method during its execution is recorded as an element of the sequence, and includes information about the object on which the method was invoked, the name of the method, and information about the associated arguments and return values. A number of mathematical functions, some of which we will introduce as needed, simplify trace manipulation, access to individual trace elements, etc. Contract requirements concerning object

Fig. 2 Observer pattern contract (partial)

interactions are then expressed in the form of appropriate conditions on the trace variables. As we will see in the next section, PTCTs will be designed to test whether the sequence of interactions that occur in a system designed using the pattern satisfy these conditions. The contract for Observer starts by listing the auxiliary concepts. Next (lines 6–8), we have a constraint that must be satisfied by the definitions of these concepts in any subcontract of Observer; we will consider this after considering the rest of the contract. Next (lines 10–11), we have the pattern invariant, the reason (or the “reward” [26]) for using the pattern, that is guaranteed provided we meet all the requirements specified in the pattern contract. players[] is the array of objects enrolled in the pattern instance in question and is another ghost variable of the contract formalism. In the case of an instance of Observer, the first player to enroll3 in any instance of the pattern is the subject; hence
The complete pattern contract will also include clauses that specify how an object enrolls to play a given role in a pattern instance, how a pattern instance is created, etc. We omit the details of these here; see [12,26]. When testing a system designed using the pattern, it is, of course, necessary that the requirements specified in these clauses of the pattern contract (specialized for the particular system) are satisfied. Hence appropriate PTCTs must be defined, or these tests must be included as part of other PTCTs.
3

123

Patterns: from system design to software testing

75

(a reference to) this object will be in players[0]. The rest of the objects in players[] will be the objects that enroll to play the Observer role (by invoking the Attach() method, this being the action required for this purpose as specified in the (elided from Fig. 2) enrollment clause for this role). Thus the invariant (lines 10–11) states that the state of the first object in players[], i.e., the state of the subject in any given instance of this pattern, is Consistent() with the states of each of the observers (currently) enrolled in this instance. Next we have (part of) the Subject role contract. First we specify the state of the subject as consisting of obs, used to store references to the currently attached observers. Next we have the specification of Attach(), one of the named methods of this role. The pre-condition requires that the attaching observer not already be attached, i.e., not have a reference to it in obs. The post-condition states4 that a reference to the object is added to obs; that the subject itself is not modified, in the sense of the auxiliary concept Modified(), and that one call to a named method of the pattern has been made during the execution of Attach(), this being to the Update() method invoked on the attaching observer. The others specification (lines 20–23) states that any other method of the class playing the Subject role should make no change in obs, and should either not modify (again in the sense of Modified()) the subject state, or must invoke the Notify() method (on the current subject). The (elided) specification of Notify() states that this method invokes Update() on each observer in obs. The Observer role contract states that the state of this role consists of the variable sub (that holds a reference to the subject to which this observer is attached); that the Update() method does not change the value of sub, and makes the state of the observer to be consistent (in the sense of the auxiliary concept Consistent()) with that of the subject. The others specification also states that it not modify sub, and leave the observer in a state that is consistent with that of the subject5 . Let us now consider the constraint specified in the contract (lines 6–8). Although the definitions of the auxiliary concepts, tailored to the needs of a particular application, will be provided by the corresponding subcontract, these definitions cannot be completely arbitrary. For example, suppose,
4

in a particular application, that s1, s2 are two states of the subject, and o1 is the state of an observer. Suppose that the definitions of Modified() and Consistent() for this application are such that each of the following is true: Consistent (s1,o1), ¬Modified (s1,s2), and ¬Consistent (s2,o1) Suppose, at some point in the execution of this system, the subject is in the state s1 and the observer in the state o1. Suppose, finally, that the subject state changes from s1 to s2. Then, according to the others specification in the Subject role, the Notify() method will not be invoked and hence Update() will not be invoked on this observer and its state would remain as o1. At this point, the current state, o1, of the observer will be inconsistent with the current state, s2, of the subject, and the invariant that the pattern was intended to guarantee will be violated although the individual methods in the individual roles, as actually implemented in the particular application, satisfy their respective specifications. The problem is that we have conflicting notions of what a sufficiently serious modification in the subject state is that requires the observers to be updated on the one hand, versus what it means for the subject state to be consistent with the observer state, on the other. The constraint specified (lines 6–8) in the contract ensures that this problem doesn’t arise. Thus while the designers of a particular system are free to tailor the definitions of the auxiliary concepts to their particular needs, they must ensure that these definitions satisfy the constraint specified in the contract. How realistic is this problem? That is, how likely is it that the software team responsible for an actual application designed using the Observer pattern would base their system on such mutually conflicting notions of Modified() and Consistent()? It depends on the size of the team in question; as the team grows larger, the likelihood of such problems seem to also grow. But even for relatively small systems, the problem can creep in during system evolution; we will see an example of this later in the paper. Unfortunately, however, specific PTCTs (or, more precisely, test cases obtained by specializing them for a particular application) cannot test these constraints because they are general conditions typically involving universal quantifiers over the states of the various objects enrolled in the pattern instance. Instead, violations of these constraints may show up as violations of, for example, the pattern invariant— although the individual methods invoked as part of the test case may satisfy their own specifications. We will see this in the discussion of the example. 3 Pattern test case templates (PTCTs) Given the requirements captured in a pattern contract, how do we develop appropriate PTCTs that can be used to test

We use the “#” notation in the post-condition to refer to the precondition value of the variable, i.e., its value when the method started execution.

5 Standard informal descriptions [11,25] of the pattern seem to suggest that the other methods of this role should not make any changes in the state of the observer. But this is unnecessarily restrictive. As the pattern contract states, all we need is that any changes in the observer be such that they leave the observer state consistent with the subject state. This is an example [12,26] of how our pattern contracts can identify dimensions of flexibility that may be missing in the standard informal descriptions.

123

76

N. Soundarajan et al.

Fig. 3 Pattern-centric testing

whether systems implemented using the pattern satisfy these requirements? It is important to stress that, given a pattern contract, construction of the corresponding PTCTs is not a mechanical activity. The purpose of the PTCTs is to help test for common mistakes in implementing the particular pattern. Clearly, which mistakes are common and which are not is a question that can be answered only on the basis of experience with systems designed using the pattern. In this section, we will present three PTCTs, the first two corresponding to the Observer pattern and the last one corresponding to the Composite pattern. These are based on our assessment, based on our own experience with implementing systems and based on the reports in the literature, of the most common mistakes in implementing this pattern. PTCTs corresponding to other patterns will similarly be based on an assessment of commonly reported mistakes made in implementing them. But these are, of course, not cast in stone. Indeed, with additional information about such mistakes, these PTCTs will be refined and additional PTCTs introduced. The main contribution of this paper, then, is the approach presented for testing systems to ensure the correct implementation of their underlying patterns—not the particular PTCTs presented. We will return to this point in the final section. Figure 3 schematically illustrates our approach to testing systems against the patterns underlying their designs. The pattern contract formally captures the requirements of the design pattern, the subcontract specifies how the pattern is specialized in the particular system. The PTCTs are designed based on the most common mistakes that are made in applying the pattern and are designed to check whether or

not the contract requirements are satisfied in those common situations. These templates must then be specialized to obtain the actual test cases corresponding to a particular system. The system tester does this specialization, as we will see in the next section, with help from JDUnit6 , a testing framework/tool that we are in the process of building, by combining the information in the contract, in the subcontract and the PTCTs, and taking account of any relevant details about the particular system. The language for defining PTCTs is intended to mirror standard Java programming syntax to support rapid practitioner adoption. Before looking at specific PTCTs, we summarize the syntax of the structure of a PTCT. A PTCT is represented as a segment of stylized Java code with interspersed asserts that, based on the pattern contract that the PTCT corresponds to, are expected to be satisfied at those points. The asserts will be based on the pre- and postconditions of the named methods as well as the other specifications in the various role contracts of the pattern contract. In addition, some of the asserts may also be based on the pattern invariant. Since the PTCT is intended to apply to all applications of the corresponding pattern, the constructor and method calls used in its definition must conform to the signatures specified in the relevant role contracts. When the actual test cases corresponding to a given application are generated from the PTCT, information in the corresponding subcontract about the mapping of named methods of the roles to particular methods of the corresponding classes and the mappings of the role-method signatures to the class-level method signatures will be used to translate the statements in the PTCT to appropriate code in the test cases. A PTCT begins with an initialization portion that creates instances of one or more role types. named methods may be invoked to bring the new player objects into states appropriate for the test case family being developed. The PTCT developer may optionally include calls to other methods, which serve as placeholders for class methods, which may be introduced at the point of test case generation. Since the precise number of other method calls required to bring an object into the state appropriate for testing a particular system varies, the PTCT syntax allows developers to specify that a single other method may be instantiated by multiple class methods, supplied at the point of test case generation. The initialization section typically concludes with a PTCT precondition. This assertion, expressed as a standard boolean expression, captures properties assumed by the PTCT body. The check ensures that the class-level constructor and method

6

The name of the framework, currently under construction, is intended to draw a parallel to JUnit, the popular unit testing framework for Java; the additional D emphasizes that the framework is used to test units of design.

123

Patterns: from system design to software testing

77

calls introduced at the point of PTCT instantiation satisfy this pre-condition. Next, a PTCT defines a body. The body portion generally follows a standard testing idiom: First, local variables are used to store the pre-conditional states of the players participating in the PTCT. Next, invocations are placed to the methods of interest, i.e., those relevant to the design pattern requirements being tested. Finally, the PTCT concludes with one or more assertions used to check that the relevant pattern requirements are satisfied. These assertions, as noted above, are based on the method and invariant specifications in the pattern contract and generally involve the post-conditional states of the participating players, as well as the pre-conditional states saved at the start of the body. There is, however, one important deviation from the standard testing idiom: Since key parts of the pattern contract involve the trace τ of method calls, the assertions contained in a PTCT will correspondingly include conditions on a special variable tau, used to record the trace of invocations originating from the PTCT. Note that the PTCT is not allowed to contain any code that directly updates this variable; the management of tau and updating it to reflect the method invocations as they happen is the responsibility of JDUnit. The one exception to this rule is that a PTCT may clear tau when the earlier invocations recorded in it are no longer of interest. In addition, JDUnit provides a number of helper functions that we may use in a PTCT; these functions allow us to obtain information about the current contents of tau. We will see several of these in the PTCTs that we discuss next. There are some important differences in detail between the trace τ used in the specifications in the pattern contract and the trace tau maintained by JDUnit. The primary reason for these differences is to enable us to construct more useful PTCTs. In particular, suppose we have a PTCT that includes a call to a method m1() followed by a call to method m2(). Thus the invocation of m1() completes before the invocation of m2() begins; i.e., the execution of neither is nested inside the other. Suppose the trace of method calls that takes place during the execution of m1() is t1 and that during m2() is t2. The trace tau maintained by JDUnit allows us to write an assert following the completion of m2() that, for example, imposes conditions on t2 based on the value of t1 or viceversa. We will see these details shortly. The main portion of a PTCT intended to test the behavior of the Attach() operation of the Subject role of the Observer pattern appears in Fig. 4. As discussed earlier, a common problem in using this pattern is not updating an observer when it initially attaches to a subject. In the PTCT, we begin by creating a subject s (line 1), and an observer o (line 2). Next, we use a “getter” method provided by JDUnit, to retrieve the pre-conditional value of s.obs before the call to Attach() (line 7), and to store this value in pre_obs. Note

Fig. 4 PTCT for subject.attach()

that JDUnit provides appropriate getter methods for each role field, including private fields.) Next, we add the attaching observer to pre_obs (line 4) to simplify the expression of the assertion check corresponding to the post-condition of Attach() (lines 9–10). We additionally store the current state of s (line 5), since this is required to check that Attach() not modify s according to the applicable definition of Modified(). A couple of points should be noted here. The getter methods are more involved than simply returning the current value of the field in question. In an actual system designed using the pattern, the class C playing a particular role R may provide a different set of fields from those specified in R’s role contract. As we will see in the next section, the subcontract for the system will specify the mapping from C’s fields to those of R. The getter methods provided by JDUnit will make use of this mapping to translate the values of C’s fields to the corresponding values of R’s fields. Similarly, the clone() operation applied to s (line 5) will create an object of whatever class plays the Subject role in the particular application. Finally, JDUnit will use the auxiliary concept definitions provided in the subcontract to evaluate assertions involving concept references, such as the one involving Modified() (line 9). We will return to these points in the next section. Let us now consider contract requirements involving traces. As we saw, in the contract formalism, each method m() has an associated trace τ , initialized to the empty sequence when the method starts execution. Information about calls made by m() to named methods are recorded in τ , with m()’s post-condition imposing necessary conditions on τ . In those cases where a PTCT includes calls to methods with trace requirements, the PTCT will include assert statements that check relevant conditions on the associated trace variables. Indeed, a PTCT may include assert statements that require particular relations to hold across multiple trace variables, each associated with methods preceding the assertion check. To enable easy expression of such asserts, we use the following approach. When the instantiated PTCT begins execution, the testing framework creates a trace object tau, initialized to the empty sequence. Each named method

123

78

N. Soundarajan et al.

invoked during the execution of the PTCT is recorded as an element of tau, and includes information about the target object, the identity of the method invoked, etc. In addition, and this is the key difference from the use of τ in the contract formalism, each element of tau itself includes the trace of methods executed during the associated call’s execution. Thus, while there is only a single tau object, it maintains a branching structure corresponding to the computation tree rooted at the PTCT. The traces contained in the individual elements of τ may be extracted using a simple accessor (“tr()”) function provide by JDUnit. Immediately prior to the call to Attach(), we clear tau (line 6). This removes the trace entries associated with the preceding constructor calls (lines 1–2), as well as any calls to named methods introduced during the specialization process in going from the PTCT to actual test cases; in the next section we will see how the specialization may introduce such calls. Hence, when control returns following the call to Attach(), tau contains a single entry, corresponding to this call. The trace of method calls that occurred during the execution of this Attach() is extracted and stored in t1 (line 8). The assert that follows imposes appropriate conditions on this trace, based on the requirements specified in the pattern contract. The first two clauses require that s._obs be appropriately updated (line 9), and that s not be Modified() (line 10). The last three clauses (lines 11–13) address trace conditions imposed on Attach(). Together, these clauses require that Attach() invoke exactly one named method, and that this call be to the Update() method of the attaching observer7 . The assert statement is, of course, based on the specification of Attach() included in the Subject role contract shown in Fig. 2. The effort involved in writing the PTCTs corresponding to a pattern is primarily in identifying the common mistakes that are made in applying the pattern and coming up with appropriate test case templates based on this. Once that has been done, the appropriate asserts to include are determined by the pattern contract. But even here, the PTCT designer may choose not to include all of the clauses specified in the role contract as the post-condition of the particular method. Even more importantly, at what points to include checks of the pattern invariant and assertions that require particular relations between the traces associated with two or more method calls appearing earlier in the PTCT, are all decisions the PTCT designer must make. And these deciWe obtain the identify of the method invoked and the target object by using the helper functions mt() and ob() provided by the JDUnit framework. These functions may be applied to individual elements of a trace in which case they return the identify of the method/target object involved in that element. Or they may be applied to a trace, in which case they return the sequence of method identities/target objects involved in the various elements of the trace. We such usage in the pattern contract in the last section.
7

Fig. 5 PTCT for subject.other()

sions will be based primarily on the common mistakes that practitioners make in using the given pattern. Note also the use of appropriate temporary objects to capture pre-conditional values referenced in various postconditions; this is common in specification-based testing given that post-conditions refer to both the state at the end of the method and at its start. The pattern invariant (lines 6–7, Fig. 2) could also have been checked as part of the assert statement. This would amount to checking that Attach() not only invokes Update() on the attaching observer, but also that the latter method appropriately updates the observer’s state to be Consistent() with the subject. Next consider the PTCT shown in Fig. 5, intended to test the behavior of Subject’s other methods. We begin by creating a subject (s) and two observers (o1, o2) (lines 1–3), and attach both observers to the subject (line 4). We then save the pre-conditional value of s.obs (line 5) and the value of s as a whole (line 6). tau is then cleared, and some other method is invoked on s (line 7). Finally, the trace associated with the other call is saved (line 8), and the behavior of the method is checked against the requirements specified in the pattern contract (lines 9–10). The assert statement requires that s.obs be unchanged, and that both o1 and o2 be Consistent() with s. In generating test cases for a particular system from this PTCT, the s.other() call will have to be replaced by calls to appropriate methods of the class playing the Subject role. Some of these methods may require additional arguments. In the next section, we will consider the problem of generating suitable values for these arguments. At this point, however, the more important issue is the mismatch between the assert statement and the requirements specified in the Observer contract. According to the contract (Fig. 2), when s.other() terminates, either the state of s must not be Modified(), or the Notify() method must have been invoked. As we have already seen, this method will in turn invoke Update() on each attached observer, bringing the objects into states consistent with the new state of the subject. Hence, the assert included in the PTCT is a test of the expected net behavior of the participating objects, i.e., it is a more “global” test of the system. There is, however, a risk in using such a PTCT, especially if these were the only ones used. If, for instance,

123

Patterns: from system design to software testing

79

an other method were to modify the state of the subject, neglect a call to Notify(), but by chance leave s in a state Consistent() with the states of o1 and o2, the design defect would go undetected. In [27], we present a scenario in which this problem manifests itself during system evolution; we summarize the example here. A class S1 plays the Subject role, and provides two fields, f1 and f2. A change in either field is considered to be a modification of the subject according to the definition of Modified() supplied in the system’s subcontract. A class O1 plays the Observer role. In an initial version of the system, O1 is interested only in the value of f1; changes in f2 are ignored. The Update() method of O1 uses an appropriate getter method to retrieve the value of f1, and then updates the observer’s state to become Consistent() with the state of the subject. The Consistent() concept is defined suitably in the subcontract. The S1 class includes a bug that omits a call to Notify() when a change in f2 occurs. A PTCT similar to the one above will not detect this defect since each observer will remain Consistent() with the subject when the other method terminates — even if f2 has changed without a corresponding call to Notify(). During system evolution, O1 is modified so that the value of f2 becomes significant. Instances of O1 record the current value of f2 associated with their corresponding subject. Hence, the definition of Consistent() is suitably modified, as is the implementation of O1’s Update() method. The new implementation retrieves the values of both f1 and f2, and updates the state of the observer appropriately. When the test case derived from the PTCT shown in Fig. 5 is executed, the assert statement will generate an error; the clause involving the Consistent() concept will be violated. The natural assumption is that the fault lies in an area affected by system evolution. In fact the defect lies in the original S1 class — in particular, the failure of its other() method to invoke Notify() when the value of f2 changes. The solution is to revise the PTCT to fully test the interaction requirements specified in the pattern contract, rather than testing for net effects. More precisely, the last two clauses of the assert in Fig. 5 (line 10) should be replaced with the following conditions:
8 9 10

Fig. 6 Composite role contract (partial)

a s s e r t ( … clauses from Fig. 5 … && (! M o d i f i e d ( pre_s , s ) || (( t1 . l e n g t h ( ) = = 1 ) & & ( t1 . m ( ) = = " N o t i f y "))

Before concluding this section, we will consider another PTCT, this one for the Composite pattern. Composite is a structural pattern [11] intended to compose objects into a tree structure to represent part-whole hierarchies and to allow clients to treat individual objects and composite objects in a uniform manner. The pattern has three roles, Component, Leaf, and Composite. Component is an abstract role with Leaf and Composite being the two kinds of components.

Patterns such as Composite, although classified as structural, also have behavioral aspects to them. Indeed, the main purpose of the Composite pattern is to ensure that each operation of a composite object is implemented by invoking the corresponding operation on each of its components and combining the results returned by these invocations appropriately to obtain the result of the original operation invocation. A small portion of the Composite pattern contract, in particular a part of the Composite role contract, appears in Fig. 6. The specification of any operation() of this role states that when this method finishes, the set of children is unchanged; that during the execution of this method, there should have been a call to the corresponding method on each child c; and that the result returned by this method should be equal to the value obtained by appropriately combining the results of these method calls invoked on the children. Since the details of precisely how these results are combined to obtain the result of the Operation() applied to the composite will vary from one application designed using this pattern to another, we use an auxiliary concept, Glue() to represent this. Thus the final clause of the post-condition of Operation() requires that the result returned by the original call is equal to the value of Glue() applied to the sequence of results in the method calls recorded in τ , this being obtained by means of the helper function, resultSeq(). Figure 7 presents a PTCT to test that the requirements specified in Fig. 6 are satisfied. We create two composites, c1 and c2, a leaf l, and create a suitable composite structure. We then invoke operation() on the top level composite structure and, assert, after the call returns that the trace created during the execution of this method is equal to the number of children of this structure, and that the result returned matches that obtained by applying the auxiliary concept Glue() on the results returned by the calls recorded in that trace. The definition of Glue() will, of course, be given in the subcontract corresponding to a particular system, for which we will specialize this PTCT to produce actual test cases. It may be worth noting that this PTCT does not completely test the specification in Fig. 6; in particular, while it does test that the

123

80

N. Soundarajan et al.

Fig. 7 PTCT for composite.operation())

number of method calls recorded in the trace t1 is what it should be according to the specification, it does not test which methods were invoked in these calls, nor on what objects. It is straightforward to modify the PTCT to include these checks and we omit the details.

4 Generating test cases Given PTCTs such as those in the last section, how do we generate actual test cases corresponding to a given system S designed using a particular pattern? Consider, for example, line 7 of Fig. 4. The method Attach() that is being invoked in this line may have an entirely different name in S. Indeed, even the classes playing the Subject and Observer roles will have their own names appropriate for the application. To illustrate these and other issues involved in generating actual test cases, we will use a simple Hospital Simulation system, parts of which appear in Fig. 8, designed using the Observer pattern. The Hospital system consists of three main classes, Patient, Nurse and Doctor. Instances of the Patient class play the Subject role; instances of the other two play the Observer role. Zero or one doctor object and zero or more nurse objects are assigned to observe patient. The variables _nurses and _doctor in the Patient class are used to keep track of the assigned doctor/nurses. A nurse object tracks the temperature (_temp) of the patient it is observing; a doctor object tracks the heart-rate (_hrtRt). The checkVitals() method updates _temp and _hrtRt as needed, and invokes notify() if needed. The Nurse and Doctor classes include an update() method to help keep track of the data about the patients being observed. They also provide a getStatus() method that returns information about the patients. Before considering how test cases corresponding to the Hospital system may be generated from the PTCTs of the last section, we have to consider the subcontract that specifies how the Observer pattern is specialized in this system; the main part of this subcontract appears in Fig. 9.

Fig. 8 Hospital simulation code

The role map for Patient as Subject specifies how the _obs of this patient viewed as as subject may be obtained from the state of the patient. Next, the method maps specify that addNurse() and setDoctor play the role of Attach() and that notify() plays the role of Notify(). The role map for Nurse as Observer illustrates a technical problem: since a nurse may be observing multiple patients, each of which is the subject in a distinct instance of Observer, how do we identify the particular patient involved with the current pattern instance? The contract formalism [12,26] provides the notion of lead object to address this. The lead object in a pattern instance is the first object that enrolls in a pattern instance, i.e., the one that is in players[0]. The method map for Nurse maps its update() (applied to the lead object) to Update() of the Observer role. The role map for Doctor as Observer is analogous. Next we have the definitions of the auxiliary concepts. According to the definition of Modified() (lines 25–27), the state of a patient is considered to be modified if either _temp value or the _hrtRt value has changed. The definition in lines 28–29 state that a nurse n’s state is consistent with that of the patient p if n has the correct information about p._temp. The definition (lines 30–31) of consistency of a doctor d and patient p is similar.

123

Patterns: from system design to software testing

81

Fig. 9 Hospital-observer subcontract

Let us now turn to the main task of this section, generating, from the PTCTs for a pattern, test cases for a particular system designed using the pattern. For any such PTCT, a (large) number of test cases may be generated. Each test case would be obtained by replacing each role object in the PTCT, such as s, o1, and o2 in the case of the PTCT in Fig. 5, by corresponding application objects, as determined by the subcontract, such as a patient, a nurse, and a doctor, respectively, in the case of the Hospital system; replacing the calls to role methods in the PTCT, such as s.Attach(o1) and s.Attach(o2), by calls to the appropriate application methods, again as determined by the subcontract, such as s.addNurse(o1) and s.setDoctor(o2), respectively; etc. While that much is straightforward, there are some important additional issues to consider. For example, it may be that the constructor functions (or other methods, for that matter) of some of the application classes require additional arguments. What values do we supply for these? The PTCT, of course, imposes no requirements on these. As far as the PTCT and the pattern are concerned, any values may be used for these additional arguments. This means that the number of possible test cases we can generate will be very large. Which of these test cases will actually be useful will depend on the system, in particular its implementation details. Hence the human tester using our approach will be required to

suitably specialize the PTCT by defining appropriate test case specializers. A key purpose of a test case specializer (TCS) is to limit the set of generated test cases, i.e., to specialize the PTCT in such a manner that the test cases that are generated are the ones most appropriate to the system under test. There are a number of dimensions in which this specialization has to be performed. First, the objects constructed in setting up the scenario for a PTCT are role objects. If R is a role of this pattern and, in the PTCT, we construct an object ob of type R, and if, according to the subcontract for the system in question the class C1 and C2 play the role R, then in an actual test case, ob may be either a C1 object or a C2 object; the TCS has to specify which it is. Second, calls in the PTCT to named role methods (such as Attach()) have to be replaced by calls to the corresponding class methods, as specified in the appropriate role maps of the subcontract. This process, by itself, does not require any additional information from the TCS – unless more than one class method plays the role method in question or, as is more frequently the case, the class method in question requires additional arguments beyond the ones expected by the role method. These values will, of course, have to be provided by the TCS. There are two important considerations to account for here. First, the application class may specify, as part of the pre-condition of the corresponding class method, some conditions that must be satisfied by these argument values. If some set of argument values generated according to the specialization in the TCS do not satisfy these conditions, that test case cannot be used. For example, in the Hospital system, we are not allowed to attach two doctor objects at the same time to a given patient object. In other words, an attempt to call addDoctor() on a patient object that already has a doctor will violate the pre-condition of this method and hence the method may behave in whatever way it chooses and this is not a bug in the system, despite the fact that, because of such behavior, an assert in the test case may fail. At the same time, we cannot always eliminate this type of problem when the test case is generated because some of these assertions may be checkable only during execution of the test case. In order to address this type of problem, JDUnit inserts additional asserts in the test cases it generates immediately prior to calls to any methods. Each of these asserts checks that the pre-condition of the method being called is satisfied. If, when a test case is executed, such an assertion is not satisfied, that is recorded in the log maintained by the system; but the assertion failure does not indicate a failure of the test case8 . The second issue that the
8

This is a distinction from the JUnit framework where every assertion violation is reported as a failure. In order to allow a different treatment for pre-condition violations, in our design of JDUnit, we use an additional method, preAssert, to specify such assertions.

123

82

N. Soundarajan et al.
8 9 10

tester designing the TCS has to be concerned about in deciding on the argument values is producing a large enough set of test cases with different values for these arguments to ensure adequate coverage of the system under test. Next, each other() call that appears in the PTCT must be converted to a call to one of the methods of the appropriate application class, other than the ones mapped to the various named role methods. In addition, as in the case considered above, if the method in question requires additional arguments, the TCS must provide these as well. And, as in the case considered above, we must account for any preconditions that may be imposed by the application class in question. The final dimension of specialization of the PTCT that the TCS must account for is concerned with a construct that may appear in a PTCT but, as it turns out, was not not used in any of the PTCTs in the last section. This is the “other*();” construct, which denotes a sequence of zero or more calls to other methods. This construct may appear in a PTCT at points where the PTCT designer wants to allow for the possibility that a number of additional calls to other methods may be needed in order to get the various objects into suitable states for continuing the test. The TCS will have to specify the sequence of method calls that must be made in place of the other*() construct. Let us now see how these considerations play out when applied to some of the PTCTs (for the Observer pattern) when we specialize them to test the Hospital system. Consider the PTCT in Fig. 4. Line 2 of this PTCT requires an observer object to be created. Suppose the TCS designer decides that this object should be a doctor. In the notation for TCS that we are designing concomitantly with the design of JDUnit, this may be expressed as:
8

A t t a c h ( o ) { @7 } = { D o c t o r ( o ): s e t D o c t o r ( o , /* addnl args */ ) || N u r s e ( o ): a d d N u r s e ( o , /* addnl args */ )};

Strictly speaking, it should not be necessary for the tester to provide this information since it is available from the subcontract (Fig. 9). But in those cases where multiple methods of the same class may play the role of a named method, the TCS will have to specify which of those must be used. This is, of course, very much needed in dealing with other() calls that appear in the PTCT since, in general, there will be many class methods that qualify as other methods. In the case of other*() calls, we must provide the sequence of method calls that should replace it and the arguments for each call in the sequence. For example, suppose the other() call in line 5 of the PTCT in Fig. 5 was an other*() call. Then we may have the following:
8 9

o t h e r *() { @7 } = { c h e c k V i t a l s (); c h e c k V i t a l s ();}

O b s e r v e r () { @2 } = D o c t o r ( /* args */ );

On the other hand, given that both Doctor and Nurse play the Observer role in this application, the TCS designer may want to create two test cases, one corresponding to each of these possibilities. This is expressed in our notation as:
8 9

O b s e r v e r () { @2 } = D o c t o r ( /* args */ ) || N u r s e ( /* args */ );

Let us consider another statement that appears in the same PTCT, the call to Attach() in line 7 of the PTCT in Fig. 4. The following specializes that call so that it is replaced by a call to the setDoctor() method:
8 9

A t t a c h ( o ) { @7 } = { s e t D o c t o r (o , /* a d d i t i o n a l args */ )}

If, as we considered above, the observer that was constructed earlier (line 2 of the PTCT) could be either a doctor or a nurse, our notation for TCS allows the tester to specify which method must be used when replacing the call to Attach() in line 7:

Here the other*() call is replaced by a sequence of two calls to checkVitals(). We can also use the “||” notation to allow for alternative sequences of calls. The detailed syntax for TCS is very much a work in progress. For example, in the situation we just considered, if the tester wanted to test a range of different sequences of other calls, listing them all explicitly would be undesirable. It would be more convenient if we could use regular expression-like notations for such situations. We intend to apply our approach to a number of patterns and systems designed using the patterns to determine the exact notational facilities that TCS must include. In addition, practitionerfriendliness will be a key consideration since it is the testers who are part of individual software teams who will be responsible for defining the TCSs for their particular systems. Before we conclude this section, one final point about the Hospital system is worth briefly discussing. Suppose, during system evolution, it is decided that the nurse objects observing a given patient object should keep track also of the medicine-level, i.e., the value of the _medLvl variable of the Patient class. It would be easy enough to modify the Nurse class to do so, perhaps by introducing a new variable, _medLevels, similar to the Nurse._vitals; and to revise Nurse.update() so that the value of _medLevels as well as _vitals is appropriately updated. While this seems simple and reasonable, if this is all we do, there will be a bug in this system! The problem is that, as it is implemented, the Patient class methods do not invoke notify() if only the _medLvl value changes. So in those situations where only this variable changes, the attached nurse objects will have incorrect information about medLvl (until the next time the patient’s _temp or _hrtRt changes, triggering a call to notify()).

123

Patterns: from system design to software testing

83

Unfortunately, however, the test cases generated using the PTCTs we have seen will not catch this error. The problem is that, according to the definition of the auxiliary concepts in Fig. 6, if only the value of _medLvl changes, the state of the patient is not considered to be modified; moreover, the state of nurse is considered to be consistent with that of the patient as long as the former has the correct information about the value of the _temp of the latter. Thus any test cases generated from the PTCT in Fig. 5 will succeed when executed. If we were to change the definition of one of these concepts but not the other, the constraint specified in the original pattern contract (lines 6–8 of Fig. 2) will be violated. Since such a problem could easily arise during system evolution, it is important to ensure that our PTCTs check that constraints on the auxiliary concepts are satisfied. At the same time, the fact that these constraints are typically universally quantified over the states of various roles makes this a challenge. This is one of the questions we intend to address in future work.

5 Related work The PTCT testing approach builds upon our prior work in pattern formalization [12,26,31]. The key conceptual elements of this work were summarized in Section 2. Several other groups have also considered improving the precision of pattern descriptions. Some have described techniques based on diagrammatic notations that extend standard UML(-like) syntax [7,21,32]. Closer to our specification approach is that of Eden et al. [9,10]; the authors describe an approach to capturing the structural properties of patterns using a logic notation. Patterns are expressed as logic formulae that specify participating classes, methods, inheritance hierarchies, and the structural relations among them. Mikkonen [19] describes a behavioral specification approach based on DisCo [14], a UNITY-like [5] action system notation. Data tuples represent objects, and guarded actions model their interactions. Helm et al. [13] describe a contract formalism that bears similarity to ours. In particular, the formalism has constructs analogous to our role contracts, auxiliary concepts, and a limited form of trace conditions. The formalism additionally provides some support for contract specialization and system-level reasoning. However, despite the effort invested in documenting patterns precisely, there appears to be relatively little prior work in testing the correctness of systems constructed according to particular pattern descriptions, either formal or informal. By contrast, this has been the focus of our work. Some readers may take note of the fact that the phrase “test(ing) patterns” appears throughout the literature. Indeed, there is at least one workshop series devoted to testing patterns [28–30], a corresponding online repository [16], and widely referenced books that detail the subject [3,18]. So how

can the work described here be new? The standard meaning of testing pattern refers to a pattern used to address a recurring problem in the context of software testing. Indeed, a test(ing) pattern is simply a special type of design pattern [17], one intended to guide the design of testing code. Our work, however, is focused on validating the correct usage of design patterns through software testing. We note that other authors have also considered applying design patterns as part of the system design process to improve testability [1,6]. Again, however, our focus has been on ensuring the correct application of patterns. It is important to note that McGregor [17,18] extends the traditional notion of a test(ing) pattern to patterns intended for use in validating design pattern implementations. The concept is similar to our use of PTCTs in that each test pattern specifies a particular scenario of application objects that interact in ways intended to check pattern requirements. However, the scenarios are not based on precise pattern specifications. Equally important, McGregor does not consider test suite specialization, nor automated generation. Still, the similarities are important. Finally, it may be useful to mention the SABER system described by Reimer et al. [22]. The system is designed to detect defects in large Java applications using static analysis; SABER operates on systems designed using particular frameworks. Correct use of these frameworks requires that certain methods be invoked in certain orders, that others not be invoked at certain points, etc. SABER attempts to identify violations of such requirements based on call sequence rules specified using an XML dialect. Given the importance of call sequence requirements in pattern-based systems, elements of SABER may be useful in detecting pattern violations statically, prior to pattern testing. We intend to explore this possibility as part of future work.

6 Conclusion We began with the observation that design patterns play a central role in software practice, influencing the design of most major software systems and class libraries. Consequently, techniques for ensuring the correctness of pattern implementations are likely to have a profound impact on the reliability of real-world systems. To this end, we focused on an approach to testing pattern implementations based on precise pattern specifications. Whereas our prior work provides the specification foundation, the work reported here provides the foundation for specification-based testing of pattern implementations. In developing the testing approach, we observed that pattern implementations follow a similar structure. This is not surprising. After all, for a designer to claim that a particular group of classes have been designed according to a given

123

84

N. Soundarajan et al.

pattern, she must have satisfied certain requirements— namely, those common to all applications of the pattern in question. It then follows that the test cases used to validate the pattern’s implementation share a common structure across systems designed using the pattern. The central motivation of our testing approach is that this common structure, applicable to all systems designed using the pattern, should be reusable. In defining the test cases for a particular system, testers should only be required to invest the incremental effort necessary to specialize the generalized structure as appropriate to the system under test. Moreover, the ability to define reusable test case structures enables the software engineering community to encode test case families that are likely to reveal the most common pattern implementation errors. We described a testing approach that satisfies these goals, and the design of a software tool used to assist in test case generation. In our approach, pattern test case templates (PTCTs) are used to define generalized test case structures reusable across applications of particular patterns. Each template is expressed using a Java-like syntax amenable to practitioner adoption; the code is written in terms of the role types and role methods specified by the corresponding pattern specification. To test the behavior of a system implemented using a particular pattern, the corresponding PTCTs must be instantiated to generate executable test cases. The instantiation process is guided by a specification of the mappings between pattern roles and system classes, as well as rules that govern system-specific specializations. JDUnit assists in the instantiation process by guiding developers in selecting the appropriate PTCTs and pattern specifications, and in introducing application-specific objects, method calls, and arguments. The output of the tool is a set of JUnit test cases used to validate the correctness of the relevant patterns as applied in the system under test9 . We conclude with a short comment on the relationship between our approach to pattern-centric testing (PCT), and test-driven-development (TDD) [2]. In TDD, test cases that capture the expected behavior of a component are written before the component is implemented. Hence, TDD dictates when test cases must be designed. On the other hand, PCT is concerned with what must be tested. One could, as dictated by TDD, develop our test cases before a system is implemented. (Of course, relevant portions of the design must be complete since otherwise we would not know which PTCTs to instantiate.) Alternatively, following a more conventional process, we may design and implement the system first, and then,
9

during testing, use PCT to generate appropriate test cases. Hence, PCT and TDD are fully compatible, but orthogonal testing principles.

References
1. Baudry B, Le Sunyé Y, Jézéquel J (2001) Towards a ‘safe’ use of design patterns to improve OO software testability. In: The 12th international symposium on software reliability engineering, IEEE Computer Society, Washington, DC, pp 324–329 2. Beck K, Gamma E (1998) Test infected: programmers love writing tests. Java Rep 3(7): 37–50 3. Binder R (1999) Testing object-oriented systems. AddisonWesley, Menlo Park 4. Buschmann F, Meunier R, Rohnert H, Sommerlad P, Stal M (1996) Pattern-oriented software architecture: a system of patterns. Wiley, New York 5. Chandy K, Misra J (1988) Parallel program design. AddisonWesley, Menlo Park 6. Dasiewicz P (2005) Design patterns and object-oriented software testing. In: The 2005 Canadian conference on electrical and computer engineering, IEEE Canada, Dundas 7. Dong J (2002) UML extenstions for design pattern compositions. In: Mingins C (ed) Proceedings of TOOLS, in special issue of journal of object technology, vol 1, issue 3, pp 149–161 8. Dong J, Alencar P, Cowan D (2001) A behavioral analysis approach to pattern-based composition. In: Proceedings of the 7th international conference on object-oriented information systems, Springer, pp 540–549 9. Eden A (2001) Formal specification of object-oriented design. In: Proceedings of the international conference on multidisciplinary design in engineering 10. Eden A (2002) LePUS: a visual formalism for object-oriented architectures. In: Proceedings of the 6th world conference on integrated design and process technology, IEEE Computer Society, pp 149–159 11. Gamma E, Helm R, Johnson R, Vlissides J (1995) Design patterns: elements of Reusable OO Software. Addison-Wesley, Menlo Park 12. Hallstrom J, Soundarajan N, Tyler B (2006) Amplifying the benefits of design patterns. In: Aagedal J, Baresi L (eds) Proceedings of the 9th international conference on fundamental approaches to software engineering (FASE), Springer, pp 214–229 13. Helm R, Holland I, Gangopadhyay D (1990) Contracts: specifying behavioral compositions in object-oriented systems. In: OOPSLAECOOP, pp 169–180 14. Järvinen H, Kurki-Suonio R (1991) Disco specification language: marriage of actions and objects. In: Proceedings of the 11th international conference on distributed computing systems, IEEE Computer Society, Los Alamitos, pp 142–151 15. Jones C (1990) Systematic software development using VDM. Prentice-Hall, Englewood Cliffs, New York 16. Marick B (2007 (date of last access)) Test patterns repository/ software testing patterns. http://www.testing.com/test-patterns/ patterns/ 17. McGregor J (1999) Testpatterns: please stand by. J Object-oriented Program 12:14–19 18. McGregor J, Sykes D (2001) A practical guide to testing objectoriented software. Addison-Wesley, Menlo Park 19. Mikkonen T (1998) Formalizing design patterns. In: Proceedings of 20th ICSE, IEEE Computer Society Press, pp 115–124 20. Rainsberger J (2005) JUnit recipes. Manning 21. Reenskaug T (1996) Working with objects. Prentice-Hall, Englewood Cliffs

A key guideline for using the JUnit framework is that the tests should be designed to test behavior rather than individual methods [20]. This also applies to our approach, except that the behaviors in question are those corresponding to the use of a particular pattern. Hence, the PTCTs typically involve multiple roles. Consequently, the instantiated test cases involve multiple system classes, by contrast to more traditional unit tests.

123

Patterns: from system design to software testing 22. Reimer D, Schonberg E, Srinivas K, Srinivasan H, Alpern B, Johnson R, Kershenbaum A, Koved L (2004) “saber”: smart analysis based error reduction. In: Proceedings of “ISSTA ’04”, ACM Press, pp 243–251 23. Riehle D (1997) Composite design patterns. In: Proceedings of OOPSLA, ACM, pp 218–228 24. Schmidt D, Stal M, Rohnert H, Buschmann F (1996) Patternoriented software architecture: patterns for concurrent and networked objects. Wiley, New York 25. Shalloway A, Trott J (2002) Design patterns explained. AddisonWesley, Menlo Park 26. Soundarajan N, Hallstrom J (2004) Responsibilities and rewards: specifying design patterns. In: Finkelstein A, Estublier J, Rosenblum D (eds) Proceedings of 26th international conference on software engineering (ICSE), IEEE Computer Society, pp 666– 675

85 27. Soundarajan N, Hallstrom J (2006) Pattern-based system evolution: a case-study. In: Zhang K, Spanoudakis G, Visaggio G (eds) Proceedings of 18th international conference on software engineering and knowledge engineering (SEKE 2006), Knowledge Systems Institute, pp 321–326 28. Testing (2001) Patterns of software testing 1 29. Testing (2001) Patterns of software testing 2 30. Testing (2001) Patterns of software testing 3 31. Tyler B, Hallstrom J, Soundarajan N (2006) A comparative study of monitoring tools for pattern-centric behavior. In: Hinchey M (ed) Procedings of 30th IEEE/NASA sofware engineering workshop (SEW-30), IEEE-Computer Society 32. Vlissides J (1998) Notation, notation, notation. C++ Report

123