Unit Testing - JUNIT

ALIN ZAMFIROIU
ALIN.ZAMFIROIU@CSIE.ASE.RO
Contents

 White box testing & Black box testing

 Test Driven Development

 Unit Testing

 Junit – Assertions

 Riht-BICEP & CORRECT

 FIRST

 TestSuite

 Double Test
White box testing & Black box testing
WhiteBox Testing

 Has many names such as:

 Clear Box Testing;
 Open Box Testing;
 Glass Box Testing;
 Transparent Box Testing;
 Code-Based Testing;
 Structural Testing.

 It is a method of testing by developers or by testers who know the internal

structure of the tested application.
WhiteBox Testing Advantages

 Testing can be commenced at an earlier stage. It is not necessarily to wait

for the GUI to be available.

 Testing is more thorough, with the possibility of covering most paths.

WhiteBox Testing Disadvantages

 Since tests can be very complex, highly skilled resources are required, with
thorough knowledge of programming and implementation.
 Test script maintenance can be a burden if the implementation changes
too frequently.
 Since this method of testing it closely tied with the application being
testing, tools to cater to every kind of implementation/platform may not
be readily available.
BlackBox Testing

 BlackBox or Behavioral Testing, is a method used to test the software

application by persons who don’t know the internal arhitecture of the
tested application.

 The tester know only the input and the output of the application.
BlackBox Testing Advantages

 The tests are made from the user’s point of view .

 Tester do not need to know programming languages or the code structure

of the application.

 Tests are done independent from the developers and have a objective
perspective.

 Test cases are designed after the specifications are complete, because
are based on these specifications.
BlackBox Testing Disadvantages

 The tests will have a small number of inputs;

 Test cases will be difficult to be designed because the tester don’t have
the specification of the application;

 Tests can be redundant with the another test made by developer.

Causes of software errors

 Programming errors – 15%

 Design errors – 30%

 Errors of specification – 55%

Test Driven Development

 Development based on tests.

 It is exactly how people think about

how to do the methods.
What is Unit Testing?

 A way of testing the code, by programmers still from the software product
development stage.

 UNIT TEST – code sequence used to test a well-defined unit of the software application
code. Usually the unit is a method.

 Unit testing is performed in a well-defined context in test specifications.

 Reasons to use Unit Tests

 Easy to write

 Tests can be written ad-hoc when you need them

 Although they are simple, they can define collections of tests – Test Suites

 They can be played automatically every time that need it (write once, use many
times)
Reasons to use Unit Tests

 There are many frameworks and tools that simplify the process of writing
and running

 Reduce the lost time for debugging and for finding bugs

 Reduce the number of bugs in the delivered or integrated code

 Increases the rate of bugs identified during the code writing phase
Reasons to not use Unit Tests

 Why do I have to test my code?

 The code written by me is correct !!!

 I do not have time for tests. I have to implement functionalities, not tests.

 There is no writed in the specification that we need to do tests.

Frameworks used for Unit Testing
Framework Programming language / scripting
JUnit Java
PHPUnit PHP
PyUnit Python
CPPUnit C++
VBUnit Visual Basic
DUnit Delphi
cfcUnit ColdFusion
HTMLUnit Html și JavaScript
Jsunit JavaScript
dotUnit .NET
NUnit C#, ASP.NET
Ruby Ruby
XMLUnit XML
ASPUnit ASP
xUnit C#
JUnit

 It is a framework that allows you to create and run tests for different
methods from the developed projects.

 The most common framework for unit code testing in JAVA.

 It is an adaptation from xUnit.

 Junit - history

 Kent Beck has developed in the 90's the first automated test tool, xUnit, for Smalltalk
 Beck and Gamma (Gang of Four) developed JUnit during a flight from Zurich to
Washington DC.
 Junit has become the standard tool for TDD-type development processes in Java
 Junit is the standard component in multiple Java IDEs (Eclipse, BlueJ, Jbuilder, DrJava,
InteliJ)
 Xunit tools have been developed for other languages as well (Perl, C++, Python, Visual
Basic, C#, …)
JUnit

 JUnit works according to two design patterns: Composite and Command.

 A TestCase class is a Command object and a TestSuite class is composed

of several TestCases or TestSuite instances.
 Junit concepts

 Fixture – set of objects used in the test

 Test Case – a class that defines the fixture set to run multiple tests
 Setup – a method / step of defining the set of objects (fixture) used for the test.
 Teardown – a method / stage of destroying the fixture after completion of the tests
 Test Suite – collection of test cases
 Test Runner – a tool used to run the tests (test suites) and to display the results.
 JUnit- Assertions
assertEquals(expected, actual)
assertEquals(expected, actual, delta)
assertSame(expected, actual)
assertNotSame(expected, actual)
assertNull(object)
assertNotNull(object)
assertTrue(condition)
assertFalse(condition)
fail(message)
assertEquals(message, expected, actual)
assertEquals(message, expected, actual, delta)
assertSame(message, expected, actual)
assertNotSame(message, expected, actual)
assertNull(message, object)
assertNotNull(message, object)
assertTrue(message, condition)
assertFalse(message, condition)
fail(message)
Junit și NUnit

JUnit NUnit
assertEquals Assert.AreEqual
assertEquals Assert.AreNotEqual
assertSame Assert.AreSame
assertNotSame Assert.AreNotSame
assertNull Assert.IsNull
assertNotNull Assert.IsNotNull
assertTrue Assert.IsTrue
assertFalse Assert.IsFalse
 JUnit3 și JUnit4 – differences

 JUnit3 needs a version of JDK newer than JDK1.2 while JUnit4 needs a version newer than
JDK5;
 in JUnit3 the test classes must be derived from the TestCase class, and in JUnit 4 it is not
necessary to inherit the TestCase class;
 in JUnit3 the name of test methods is built on the testAAA format; so all test methods
contain in their name the test word; in JUnit4 methods name is not important, but the
methods that are run as tests have the @Test annotation;
 in JUnit4 it is possible to use the adnotation @Ignore, if a test should be avoided; in JUnit3 that
test should be deleted or commented.
Test Case
Unit testing - Junit skeleton
Unit testing - Junit skeleton
 Unit testing - Junit skeleton

 Annotations used in JUnit4 for automated methods:

 @BeforeClass for setUpBeforeClass() method;

 @AfterClass for tearDownAfterClass() method;

 @Before for setUp() method;

 @After for tearDown() method.

 Because in JUnit3 ,are no annotations name of the methods setup () respectively

teardown () are required.
Unit testing - Junit skeleton

JUnit4 JUnit5 - Jupiter

@BeforeClass @BeforeAll
@AfterClass @AfterAll
@Before @BeforeEach
@After @AfterEach
Unit testing - Assertions

 Create class Utils in Eclipse.

 Create unit tests for the methods of this class, using JUnit4.
Unit testing
Unit testing - Assertions

assertEquals(expected, actual)
assertEquals(expected, actual, delta)
assertSame(expected, actual)
assertNotSame(expected, actual)
Test the “sum” method. assertNull(object)
assertNotNull(object)
assertTrue(condition)
assertFalse(condition)
fail(message)
Unit testing - Assertions - assertEquals
Unit testing - Assertions

assertEquals(expected, actual)
assertEquals(expected, actual, delta)
assertSame(expected, actual)
assertNotSame(expected, actual)
Test the “division” method.
assertNull(object)
assertNotNull(object)
assertTrue(condition)
assertFalse(condition)
fail(message)
Unit testing - Assertions - assertEquals

That method can fail. So we have to test that, also.

Unit testing - Assertions - fail
Unit testing - Assertions - fail

 Because the test fail, we have to correct the method.

 Now the test should pass.

 Unit testing - Assertions

assertEquals(expected, actual)
assertEquals(expected, actual, delta)
assertSame(expected, actual)
assertNotSame(expected, actual)
Test the “isEven” method.
assertNull(object)
assertNotNull(object)
assertTrue(condition)
assertFalse(conditin)
fail(message)
Assertions – assertTrue / assertFalse
 Unit testing - Assertions
Test the “nEvenNumbers” method.
assertEquals(expected, actual)
assertEquals(expected, actual, delta)
assertSame(expected, actual)
assertNotSame(expected, actual)
assertNull(object)
assertNotNull(object)
assertTrue(condition)
assertFalse(condition)
fail(message)
Assertions – assertNull / assertNotNull
Assertions – assertNull / assertNotNull

 This method also should be modified, because for n=0, we will have an
empty list, not a null value.
Assertions

 These assertions are only a small part of the all assertions that exists, but are
the most used.
Right-BICEP principle
Right-BICEP

 Now that we know how to do tests, all we have to do is to know what tests we have
to do.

 People with a lot of experience know directly (from their own experience) what they
have to test.

 People with less experience need to follow certain directions or certain principles.

 The best known test principle is Right-BICEP.

Unit testing - Right-BICEP

 RIGHT – are the results right?

 B – are all the Boundary conditions correct?
 I – check Inverse relationship;
 C – Cross-Check results using other means;
 E – force Error condition to happen;
 P – are Performance characteristics verified?
Unit testing - Right

 Every time when we test a method, the first thing that should be veryfied is
if that method delivers the right results.
 That is why the first direction is to check the correctness of the results.
 This verification is made in accordance with the specifications of the
developed project.
 For the Utils.java class it is necessary to check :
 if the sum is correct calculated.
 if the division is correct determinate.
Unit testing - Right
Unit testing - Boundary

 Problems usually appear at the edges, so we need to be careful to test

the methods for the boundaries of the ranges.

 For each method, we have to determine the interval in which input

parameter values can be, as well as the range of the result provided by
the method.
Unit testing - Boundary

 Once these limits have been determined, exact tests are performed for
those values.

 Boundary tests do not require testing outside these values but checking
the correctness for these values - limit values.

 There are usually lower limits and higher limits. Tests are done for both
situations.
Unit testing - Boundary

 For the Utils.java class it is necessary to check

 The division to 1 it will be the numerator value?.
 The division of a number to that number itself, returns 1?
Unit testing - Boundary
Unit testing - Boundary

 In order to identify the extreme limits more easily, we must use the
CORRECT principle :
 C – Conformance;
 O – Ordering;
 R – Range;
 R – References;
 E – Existence;
 C – Cardinality;
 T – Time.
Unit testing - Inverse relationship

 Certain methods can be tested by applying the inverse rule: starting from
the result it must reach the same input from which it initially started..

 Not applicable for all methods. It usually applies to mathematical

methods.
Unit testing - Inverse relationship

 Also for the databases it can be checked whether an insert was made by
the reverse operation: select.

 For the Utils.java class to check

 Check the sum in inverse way – by difference;
Unit testing - Inverse relationship
Unit testing - Cross-Check

 For each method, try testing it by using another method.

 Usually there are several ways to solve a problem. Thus, another method
for solving the same problems for testing the newly implemented method
can be used
Unit testing - Cross-Check

 This situation is possible when the implemented method was designed to

increase productivity, or if the old method was consuming resources or
time.

 Testing the new method is done through the old method, even if it
consumes more resources.
Unit testing - Cross-Check

 For the Utils.java class to check:

 The division method can be verified by / operator.
Unit testing - Cross-Check
Unit testing - Error conditions

 Probably the ugliest scenario for an application is to crash. That's why

when we test each method unitally, we also need to test the situations in
which the application might crash.

 If we have studied the extreme boundaries for input value ranges or

resulting values, testing for error provisioning should use values outside
these ranges.
Unit testing - Error conditions

 Testing for forcing errors is done on all methods. All methods have at least
one situation where they will provide error. Testing is done for these
situations and it is verified that the method treats that case and throws or
delivers an exception.
Unit testing - Error conditions

 For the Utils.java class to check:

 What happening if we try the division method with denominator=0?
 What is happening if the parameter of “nEvenNumbers” method is -2?
Unit testing - Error conditions
Unit testing - Error conditions
Unit testing - Performance

 For different methods, it is possible to test how well the method is

performing.

 Besides testing the correctness of the results of the methods, it is very

important to check and perform the results with which they are obtained.
Unit testing - Performance

 Performance verification is done both in terms of consumed resources and

as time required to achieve the results.

 Performance testing is performed when the input or result of a method is

represented by a list or a very large number of elements, and these values
can increase a lot.
Unit testing - Performance

 For JUnit4 to test the time it is used the next adnotation: @Test(timeout=100)

 For the Utils.java class to check:

 It is enough 1 second to obtain the first 1.000.000 even numbers?
 but for 1.000.000.000 of numbers?
Unit testing - Performance
What else to test?
CORRECT principle
CORRECT

 C – Conformance;
 O – Ordering;
 R – Range;
 R – Reference;
 E – Existence;
 C – Cardinality;
 T – Time.
CORRECT

 Each subprinciple has a question that should be in the mind of the tester.

 This principle is used also to helps the testers for the boundery conditions
Conformance

 It is also known as
 Type testing
 Compliance testing
 Conformity assessment
Conformance

 It is applied a lot of domains where something should meet some specific

standards.

 Usually for any input and for any output it must be verified that it conforms
to a format or to a standard.
Conformance

 Tests can be done to check what happens if the input data does not
conform to the format or to see if the result obtained conforms to the
project specification format.
Conformance

 For the Person class we have to test if the CNP atribute has 13 characters.
Conformance
Ordering

 Order tests are specific to lists but not only.

 In the case of lists, check if the order of the items is the one that you want.

 You can also test the behavior of the method if it receives some
parameters in another order or a list of items in a different order than
expected.
Ordering

 We can check if the result of the nEvenNumbers method is a list in the

correct order, if the second one is bigger than the first one; the third one is
bigger than the second and so on.
Ordering
Range

 For both input and output values, certain intervals are set. These intervals
need to be checked.

 Several intervals are set for certain methods. This will test for all these
intervals.
Range

 All functions that have an index must be tested for range because that
index has a well-established range.
 Usually it is necessary to check:
 The start and end value for the index are the same;
 The first element is larger or smaller than the last element;
 What happens if the index is negative;
 What happens if the index is higher than the upper limit;
 The number of items is not the same as the one you want;
 Etc.
Range

 For the nEvenNumbers () method, it can be checked if length of the returned

list is required number.
Range
Reference

 Certain methods depend on external things or external objects to these

methods. These elements need to be verified and controlled.

 For example:
 A web application requires the user to be logged in;
 A pop stack works if there are elements in the stack;
 Etc.
Reference

 These elements are also called preconditions.

 Preconditions for that the method to work normally.

 These tests are performed using test doubles (stub, fake, dummy, mock).
Existence

 “does some given thing exist?”

 We need to ask what happens to the method if a parameter does not

exist, if it is null or if it is 0.

 Also for software systems that work with files or with internet connection, it
is necessary to check the existence of these files or the availability of
internet connection. Otherwise, the application does not need to crack
but must behave normally.
Existence

 Make sure your method can stand up to

nothing.

 It is similar to the error conditions from

Right-BICEP.
Cardinality

 0-1-n Rule

 It is simlar to the Existence tests and Range tests.

 We have to verify if the method has 0 elements, 1 element or n elements.

 If it works for 2, 3 or 4 elements is considered that it will work for more

elements.
Cardinality

 For the nEvenNumbers() method we have to check the situations:

 0 even numbers;
 1 even number;
 n even numbers, where n in {2,3,4….}.
Time

 It is similar to the Right-BICEP performance test.

 It can also be tested if the call template is respected. Similar to the

Template design pattern.

 For example, to call the logout method(), you must first call the login
method() and other examples.
F.I.R.S.T.
F.I.R.S.T

 “ For unit tests to be useful and effective for your programming team you
need to remember to make them FIRST.”
F.I.R.S.T

 Fast
 Isolated/Independent
 Repeatable
 Self-Validating
 Timely
Fast

 The developed test should be fast because if we have too many tests we
don’t have to wait to much time when we run them.
Isolated

 Single responsibility (SOLID)

 “Each unit test should have a single reason to fail.”

Isolated

 When a test fail, the developer should not debug to identify what is wrong
and where.

 The test should be isolated and to say exactly where is the problem and
what problem.
Repeatable

 The obtained results should be the same indifferent of the number of test runs.

 Tests should run repeatedly without any other interventions.

Self-Validating

 The confidence in the implemented tests.

 If the tests pass, the developer should have high confidence that the code
is correct without errors.

 If a test fail the developer must have high confidence that the code
should be improved.
Timely

 When we have to implement the tests for our method?

 When do we consider that we have done all tests?

TestSuite
Unit testing – TestSuite

 Why do we need suites of tests?

 In this moment we have a lot of testCases and we have to run only few
test from these testCases.

 With suites we can group some tests from different testCases and to run
only these tests.
Unit testing – TestSuite
Custom TestSuite – JUnit 4

 The CustomSuite class or interface is being implemented.

 For each test that you want to be part of that suite, add the annotation :
 @Category(CustomSuite.class)
Custom TestSuite – JUnit 4

 To create a custom suite, all the desired categories are included.

 A single category is included in this case. Tests in this category are found in
two TestCases UtilsTest and PersonTest :

@RunWith(Categories.class)
@IncludeCategory(CustomSuite.class)
@SuiteClasses({ UtilsTest.class, PersonTest.class })
public class NewSuiteTests {
}
Custom TestSuite – JUnit 4

 We can also exclude some categories from our suite.

 In this way we can create custom suites, based only on these categories.
Double test
Double test

 Every time when we test a method, we try to test the correctness of this
method.

 There is a problem for situations where our method uses external objects,
external links, or external method calls.

 These objects, links, or methods can influence the results obtained using
the method that we want to test.
Double test

 This is the way to test the method, but with a control over the external
references.
 REFERENCE (CORRECT)

 For this, test doubles are used.

Double test

 In automated testing, it is common to use objects that look and behave

like their production equivalents, but they are actually simplified. This
reduces complexity, allows verification of code independent of the rest of
the system, and sometimes it is even necessary to perform self-validation
tests. A generic term used for these objects is a double test.

 A double test is simply another object that fits the necessary collaborator
interface and can be replaced in its place. There are several types of test
doubles.
 Dummy object

 Dummy object – an object that respects the interface but the methods do nothing or
return 0 or null.

 When we have to use the real object, actually we use a dummy object.

 These doubles are used when we don’t have to call the methods from that object.
Because they are doing nothing.
Dummy object
Stub

 Stub – in contradistinction to Dummies, methods from a Stub will return

conserved / hardcoded responses.

 In this way we can use these objects with real calls.

Stub

https://blog.pragmatists.com/test-doubles-fakes-mocks-and-stubs-1a7491dfa3da
Stub
Fake

 Fake – is an object that behaves like a real one but has a simplified version.

 Usually for a fake we can set what the value should it return. It will not be a
hardcoded value.
Fake

https://blog.pragmatists.com/test-doubles-fakes-mocks-and-stubs-1a7491dfa3da
Spy

 Spy– is a Stub or Fake that manages and counts the number of calls.
Mock

 Mock – different from all the others, but works similar.

https://blog.pragmatists.com/test-doubles-fakes-mocks-and-stubs-1a7491dfa3da
Mock testing

 Mock testing – It is used when we want the tested method not to be influenced by
external references.

 Mock object – an object that simulates the behavior of a real object, but in a controlled
manner.

 Frameworks used like Mockito, EasyMock, etc.

Mockito

 Download the jar and add to the current project.

 If you do not find it on google:

https://mvnrepository.com/artifact/org.mockito/mockito-all/1.10.19
Mockito

 A mock object is created based on a real class :

 Set behavior for desired methods:

Mockito

 Methods:
 doReturn()
 doAnswer()
 when()
 thenReturn()
 thenAnswer()
 thenThrow()
 doThrow()
 doCallRealMethod()
Mockito
Test data files

 There will be a single TestCase with the assert method called in a loop.

 For every set of test data in the file, call the assert method.

 What you've improved on this code?

Test data files

 The file, or stream is considered a fixture and therefore must be opened in

setUp and closed in tearDown.
Integration tests and Unit tests
Integration tests and Unit tests
References

 https://github.com/ghsukumar/SFDC_Best_Prac
tices/wiki/F.I.R.S.T-Principles-of-Unit-Testing
 https://pragprog.com/magazines/2012-01/unit-
tests-are-first
 http://agileinaflash.blogspot.com/2009/02/first.
html