Chapter 5

Boundary Value Testing

In Chapter 3, we saw that a function maps values from one set (its domain) to values in another set
(its range) and that the domain and range can be cross products of other sets. Any program can be
considered to be a function in the sense that program inputs form its domain and program outputs
form its range. In this and the next two chapters, we examine how to use knowledge of the func-
tional nature of a program to identify test cases for the program. Input domain testing (also called
“boundary value testing”) is the best-known specification-based testing technique. Historically,
this form of testing has focused on the input domain; however, it is often a good supplement to
apply many of these techniques to develop range-based test cases.
There are two independent considerations that apply to input domain testing. The first asks
whether or not we are concerned with invalid values of variables. Normal boundary value testing
is concerned only with valid values of the input variables. Robust boundary value testing consid-
ers invalid and valid variable values. The second consideration is whether we make the “single
fault” assumption common to reliability theory. This assumes that faults are due to incorrect val-
ues of a single variable. If this is not warranted, meaning that we are concerned with interaction
among two or more variables, we need to take the cross product of the individual variables. Taken
together, the two considerations yield four variations of boundary value testing:

◾◾ Normal boundary value testing

◾◾ Robust boundary value testing
◾◾ Worst-case boundary value testing
◾◾ Robust worst-case boundary value testing

For the sake of comprehensible drawings, the discussion in this chapter refers to a function, F,
of two variables x1 and x 2. When the function F is implemented as a program, the input variables
x1 and x 2 will have some (possibly unstated) boundaries:

a ≤ x1 ≤ b

c ≤ x2 ≤ d

79
© 2010 Taylor & Francis Group, LLC
80  ◾  Software Testing

x2

x1
a b

Figure 5.1  Input domain of a function of two variables.

Unfortunately, the intervals [a, b] and [c, d] are referred to as the ranges of x1 and x 2, so right
away we have an overloaded term. The intended meaning will always be clear from its context.
Strongly typed languages (such as Ada® and Pascal) permit explicit definition of such variable
ranges. In fact, part of the historical reason for strong typing was to prevent programmers from
making the kinds of errors that result in faults that are easily revealed by boundary value testing.
Other languages (such as COBOL, FORTRAN, and C) are not strongly typed, so boundary value
testing is more appropriate for programs coded in these languages. The input space (domain) of
our function F is shown in Figure 5.1. Any point within the shaded rectangle and including the
boundaries is a legitimate input to the function F.

5.1 Normal Boundary Value Testing

All four forms of boundary value testing focus on the boundary of the input space to identify
test cases. The rationale behind boundary value testing is that errors tend to occur near the
extreme values of an input variable. Loop conditions, for example, may test for < when they
should test for ≤, and counters often are “off by one.” (Does counting begin at zero or at one?)
The basic idea of boundary value analysis is to use input variable values at their minimum, just
above the minimum, a nominal value, just below their maximum, and at their maximum. A
commercially available testing tool (originally named T) generates such test cases for a prop-
erly specified program. This tool has been successfully integrated with two popular front-end
CASE tools (Teamwork from Cadre Systems, and Software through Pictures from Aonix [part
of Atego]; for more information, see http://www.aonix.com/pdf/2140-AON.pdf ). The T tool
refers to these values as min, min+, nom, max–, and max. The robust forms add two values,
min– and max+.
The next part of boundary value analysis is based on a critical assumption; it is known as the
“single fault” assumption in reliability theory. This says that failures are only rarely the result of
the simultaneous occurrence of two (or more) faults. The All Pairs testing approach (described in
Chapter 20) contradicts this, with the observation that, in software-controlled medical systems,
almost all faults are the result of interaction between a pair of variables. Thus, the normal and
robust variations cases are obtained by holding the values of all but one variable at their nominal

© 2010 Taylor & Francis Group, LLC

 Boundary Value Testing  ◾  81

x2

x1
a b

Figure 5.2  Boundary value analysis test cases for a function of two variables.

values, and letting that variable assume its full set of test values. The normal boundary value analy-
sis test cases for our function F of two variables (illustrated in Figure 5.2) are

{<x1nom, x 2min>, <x1nom, x 2min+>, <x1nom, x 2nom>, <x1nom, x 2max–>, <x1nom, x 2max>, <x1min, x 2nom>, <x1min+,
x 2nom>, <x1max–, x 2nom>, <x1max, x 2nom>}

5.1.1 Generalizing Boundary Value Analysis

The basic boundary value analysis technique can be generalized in two ways: by the number of
variables and by the kinds of ranges. Generalizing the number of variables is easy: if we have a
function of n variables, we hold all but one at the nominal values and let the remaining variable
assume the min, min+, nom, max–, and max values, repeating this for each variable. Thus, for a
function of n variables, boundary value analysis yields 4n + 1 unique test cases.
Generalizing ranges depends on the nature (or more precisely, the type) of the variables them-
selves. In the NextDate function, for example, we have variables for the month, the day, and the
year. In a FORTRAN-like language, we would most likely encode these, so that January would
correspond to 1, February to 2, and so on. In a language that supports user-defined types (like
Pascal or Ada), we could define the variable month as an enumerated type {Jan., Feb., …, Dec.}.
Either way, the values for min, min+, nom, max–, and max are clear from the context. When a
variable has discrete, bounded values, as the variables in the commission problem have, the min,
min+, nom, max–, and max are also easily determined. When no explicit bounds are present, as
in the triangle problem, we usually have to create “artificial” bounds. The lower bound of side
lengths is clearly 1 (a negative side length is silly); but what might we do for an upper bound? By
default, the largest representable integer (called MAXINT in some languages) is one possibility;
or we might impose an arbitrary upper limit such as 200 or 2000. For other data types, as long as
a variable supports an ordering relation (see Chapter 3 for a definition), we can usually infer the
min, min+, nominal, max–, and max values. Test values for alphabet characters, for example,
would be {a, b, m, y, and z}.
Boundary value analysis does not make much sense for Boolean variables; the extreme values
are TRUE and FALSE, but no clear choice is available for the remaining three. We will see in

© 2010 Taylor & Francis Group, LLC

82  ◾  Software Testing

Chapter 7 that Boolean variables lend themselves to decision table-based testing. Logical variables
also present a problem for boundary value analysis. In the ATM example, a customer’s PIN is a
logical variable, as is the transaction type (deposit, withdrawal, or inquiry). We could go through
the motions of boundary value analysis testing for such variables, but the exercise is not very sat-
isfying to the tester’s intuition.

5.1.2 Limitations of Boundary Value Analysis

Boundary value analysis works well when the program to be tested is a function of several
independent variables that represent bounded physical quantities. Mathematically, the vari-
ables need to be described by a true ordering relation, in which, for every pair <a, b> of values
of a variable, it is possible to say that a ≤ b or b ≤ a. (See Chapter 3 for a detailed definition of
ordering relations.) Sets of car colors, for example, or football teams, do not support an order-
ing relation; thus, no form of boundary value testing is appropriate for such variables. The key
words here are independent and physical quantities. A quick look at the boundary value analysis
test cases for NextDate (in Section 5.5) shows them to be inadequate. Very little stress occurs on
February and on leap years. The real problem here is that interesting dependencies exist among
the month, day, and year variables. Boundary value analysis presumes the variables to be truly
independent. Even so, boundary value analysis happens to catch end-of-month and end-of-year
faults. Boundary value analysis test cases are derived from the extrema of bounded, independent
variables that refer to physical quantities, with no consideration of the nature of the function,
nor of the semantic meaning of the variables. We see boundary value analysis test cases to be
rudimentary because they are obtained with very little insight and imagination. As with so
many things, you get what you pay for.
The physical quantity criterion is equally important. When a variable refers to a physical quan-
tity, such as temperature, pressure, air speed, angle of attack, load, and so forth, physical bound-
aries can be extremely important. (In an interesting example of this, Sky Harbor International
Airport in Phoenix had to close on June 26, 1992, because the air temperature was 122°F. Aircraft
pilots were unable to make certain instrument settings before takeoff: the instruments could only
accept a maximum air temperature of 120°F.) In another case, a medical analysis system uses
stepper motors to position a carousel of samples to be analyzed. It turns out that the mechanics
of moving the carousel back to the starting cell often causes the robot arm to miss the first cell.
As an example of logical (vs. physical) variables, we might look at PINs or telephone numbers.
It is hard to imagine what faults might be revealed by testing PIN values of 0000, 0001, 5000,
9998, and 9999.

5.2 Robust Boundary Value Testing

Robust boundary value testing is a simple extension of normal boundary value testing: in addition
to the five boundary value analysis values of a variable, we see what happens when the extrema
are exceeded with a value slightly greater than the maximum (max+) and a value slightly less than
the minimum (min–). Robust boundary value test cases for our continuing example are shown in
Figure 5.3.
Most of the discussion of boundary value analysis applies directly to robustness testing, espe-
cially the generalizations and limitations. The most interesting part of robustness testing is not
with the inputs but with the expected outputs. What happens when a physical quantity exceeds its

© 2010 Taylor & Francis Group, LLC

 Boundary Value Testing  ◾  83

x2

x1
a b

Figure 5.3 Robustness test cases for a function of two variables.

maximum? If it is the angle of attack of an airplane wing, the aircraft might stall. If it is the load
capacity of a public elevator, we hope nothing special would happen. If it is a date, like May 32, we
would expect an error message. The main value of robustness testing is that it forces attention on
exception handling. With strongly typed languages, robustness testing may be very awkward. In
Pascal, for example, if a variable is defined to be within a certain range, values outside that range
result in run-time errors that abort normal execution. This raises an interesting question of imple-
mentation philosophy: is it better to perform explicit range checking and use exception handling
to deal with “robust values,” or is it better to stay with strong typing? The exception handling
choice mandates robustness testing.

5.3 Worst-Case Boundary Value Testing

Both forms of boundary value testing, as we said earlier, make the single fault assumption of reli-
ability theory. Owing to their similarity, we treat both normal worst-case boundary testing and
robust worst-case boundary testing in this subsection. Rejecting single-fault assumption means
that we are interested in what happens when more than one variable has an extreme value. In elec-
tronic circuit analysis, this is called “worst-case analysis”; we use that idea here to generate worst-
case test cases. For each variable, we start with the five-element set that contains the min, min+,
nom, max–, and max values. We then take the Cartesian product (see Chapter 3) of these sets to
generate test cases. The result of the two-variable version of this is shown in Figure 5.4.
Worst-case boundary value testing is clearly more thorough in the sense that boundary value
analysis test cases are a proper subset of worst-case test cases. It also represents much more effort:
worst-case testing for a function of n variables generates 5n test cases, as opposed to 4n + 1 test
cases for boundary value analysis.
Worst-case testing follows the generalization pattern we saw for boundary value analysis. It also
has the same limitations, particularly those related to independence. Probably the best application
for worst-case testing is where physical variables have numerous interactions, and where failure of
the function is extremely costly. For really paranoid testing, we could go to robust worst-case testing.
This involves the Cartesian product of the seven-element sets we used in robustness testing resulting
in 7n test cases. Figure 5.5 shows the robust worst-case test cases for our two-variable function.

© 2010 Taylor & Francis Group, LLC

84  ◾  Software Testing

x2

x1
a b

Figure 5.4  Worst-case test cases for a function of two variables.

x2

x1
a b

Figure 5.5 Robust worst-case test cases for a function of two variables.

5.4 Special Value Testing

Special value testing is probably the most widely practiced form of functional testing. It also is
the most intuitive and the least uniform. Special value testing occurs when a tester uses domain
knowledge, experience with similar programs, and information about “soft spots” to devise test
cases. We might also call this ad hoc testing. No guidelines are used other than “best engineering
judgment.” As a result, special value testing is very dependent on the abilities of the tester.
Despite all the apparent negatives, special value testing can be very useful. In the next section,
you will find test cases generated by the methods we just discussed for three of our examples. If
you look carefully at these, especially for the NextDate function, you find that none is very satis-
factory. Special value test cases for NextDate will include several test cases involving February 28,
February 29, and leap years. Even though special value testing is highly subjective, it often results
in a set of test cases that is more effective in revealing faults than the test sets generated by bound-
ary value methods—testimony to the craft of software testing.

© 2010 Taylor & Francis Group, LLC