Program Verification, Problems and unsolved

issues in unit testing

Jorge Teran
Universidad Mayor de San Andres
jteran@umsa.bo
February 8, 2015

Contents
1 Introduction 1

2 Problem Statement 2

3 Verifying and testing the program 3

4 Unsolved questions 4

5 Conclusions 4

Abstract
Unit testing gives the possibility of automating the test process
of small unit of code, e.g., a method or a class. This article shows
different problems, incomplete tests, specification errors, generating
test data and how to address these problems.

1 Introduction
To automate the testing a developer writes a piece of code called unit
test. This test unit executes a specific functionality in the code to be tested.
Most of the This test will execute a small unit of code.

1
 The purpose of unit tests is to ensure that the code works as intended.
When you modify the code for fixing a bug or extending the functionality
unit testing will help to avoid many manual testings.
In this article we will use Junit that is a popular testing framework for
Java. The issues and principles shown apply to any automating testing.

2 Problem Statement
Suppose our program has to read three integers from the screen perform
a calculation and show the result. To make the testing possible the first
thing is to divide the program in to parts, one that reads the input data
and a method that makes the calculations. The testing must be made to
the method that performs the calculations, passing values and verifying the
results.
The calculations requested are to evaluate the results of x in the following
expression:
ax2 + bx + c = 0
From math we know there are two answers is:
√
−b + b2 − 4ac
x1 =
2a
and √
−b −b2 − 4ac
x2 =
2a
For simplicity we will analyze only the case that has integer solutions
avoiding imaginary complex numbers.
To solve the problem we code the formula in the program Eq1.java:

To start the unit testing we first need some input data for the method
solve and the correct answer for this data. The program reads the equation
parameters a, b, c and calls the solve method and receives the answer in a
array with the two values that are the results we expect.
For example, if we send the values (1, 0, −1) this means that we want solve
2
1x + 0x − 1 = 0. Solving this we find the two solutions x1 = 1, x2 = −1.
To create the testing program we send this values and verify the result is
correct. The program Eq1Test performs the unit testing.

2
3 Verifying and testing the program
We run the test program and find that no errors are reported. Are the
tests programed sufficient for this method? Do we need more test data?
The trivial error is to assign the result to an integer. It should be a double
variable to allow floating point numbers, but for this problem lets consider
that the answers are integer numbers.
The first thing that we see is that the test data is small. For this reason
we write more test data so it covers a wider range of values. To perform this
task we need to remember to some mathematical properties. The equation
(wx+n)(x+m) will yield the following expression wx2 +wxm+nx+n+m =
wx2 + x(wm + n) + (n ∗ m). This is a general case where the solutions are
x = −m and x = −n/w. We could simplify this tests putting m as a multiple
of w.
Instead of handwriting the test data we develop a program that writes
the code required in the test program to create more test cases.

The output would be something like this

int x35[];
int[] ans35={2,-2};
x35=test.solve(2,8,8);
assertArrayEquals(x35, ans35);

Then you could copy and paste to the testing program or write the code
directly there. This test data generate cases with a = 2 and b2 − 4ac > 0.
Other values like a = 0 are no generated.
Running the tests we find that some errors appear. This means that the
tests we developed at the beginning were not enough. Where do this errors
come from? Reviewing the code we see that the formulas are correct and
according to the specification.
As you can see we did not specify if the results should have some order
for example the lowest value first. Even if the results are correct the program
can produce an error e.g [1.-1] is not the same as [-1.1]. Where does this error
come from? Is this an specification or testing error? Do we need to review
and change the problem statement?

3
4 Unsolved questions
How do we know if are tests are complete or sufficient? In our case it is
quite simple so see if all possibilities were taken in account in other situation
it could be very complicated.
Is our test data correct? This question takes us to think if our test data
needs to be tested for correctness. How do we do this?
How do we test our specification?. They could also be wrong or be in-
complete as the shown example.

5 Conclusions
Testing can be a very complicated task in software development. Many
things have to be taken in account. To have only some tests is not enough.
Generating test data in an automatic form can help create much more test
data than manually.
Reviewing how the test data is created is a must. This can help to
determine if the tests are complete and accurate.