You are on page 1of 20

JayView

2007 No. 1 Volume 13
2007 No. 1
Volume 13
Java ME Testing Done Right Signing MIDlets - the whys and hows Essential Java Generics Problems
Java ME Testing Done Right
Signing MIDlets - the whys and hows
Essential Java Generics
Problems with object creation
Neo – a netbase
man Interview: in the Martin know Fowler –
JayView 2007 No. 1 Volume 13 Java ME Testing Done Right Signing MIDlets - the whys
Java ME Testing Done Right
Java ME Testing
Done Right

2

JayView

Publication Offi cer

Thomas Dagsberg, MD Jayway AB

Editors

Leif Uller, Björn Granvik, Darius Katz, Patrik Nilsson

Jayway

You will fi nd the addresses and telephone numbers of the various offi ces on the back.

Cover Picture

Capo da Roca, Portugal

Björn Granvik

Layout and Graphic Design

Roger Petersson

Subscription of JayView

Send an email with your address to info@jayway.se

  • I s there a right or wrong way to unit test applications? We believe so. Espe- cially when it comes to testing Java ME applications. Down-scaled copies of our faithful testing frameworks are popping up everywhere in the Java ME

community. But has anyone dared to ask the question: Why?

The Problem

Java Micro Edition (Java ME) is the most ubiquitous application platform for mobile devices. However, testing Java ME applications is not a simple task. The constraints on the Virtual Machine (VM), the limited resources and the not-always-so-well- designed APIs are all contributors to this statement. The limited environment has ”forced” application developers to write their code as compact as possible. On many an occasion this has led to an unsound application design that isn’t test friendly. In a test friendly design, the collaborators of an object can easily be stubbed. Such a design is often called a testable design. There are architectural patterns such as de- pendency injection that can be used to achieve a testable design. Even though this is mostly used in enterprise applications, it can be used with most modern program- ming languages and environments. The ability to stub collaborating objects is great when testing code. We can create fake or ”mock” instances of our collaborating objects and set up an expected behav- ior for them. By doing this, we can isolate the code being exercised and verify that the behavior of the collaborating objects was met, i.e. verify that the code under test actually did what we intended it to do. This is known as unit testing based on mock objects. Those who are inexperienced with this kind of testing can fi nd some good articles on the subject in the References section located at the end of this article. In the Java SE world, there are excellent tools like JUnit and EasyMock for creat- ing unit tests based on mock objects. Since Java ME doesn’t support refl ection, we have lighter versions of JUnit, such as J2MEUnit, JMUnit and Mobile JUnit. As the Connected Limited Device Confi guration (CLDC) VM doesn’t support dynamic object creation, we can’t use EasyMock and have to write the mock objects our- selves. Even if you have a testable design, there are more issues that make unit testing dif- fi cult in Java ME:

• Creating mock objects is tedious and error prone. There are tools that can help you generate code for your mock objects, but you have to regenerate these if you change your objects, otherwise you may encounter version confl icts. This is why dynamic frameworks like EasyMock have gained such popularity. Running unit tests on the target platform takes time. Even if it only takes a minute to install and run some tests, it may be enough to make you not want to run the tests every time you change your code. Running JUnit tests in Eclipse takes a cou- ple of seconds. Furthermore, the limited environment puts a constraint on how many tests you can run on your Java ME device. This forces you to split your test cases between several test suites, creating even longer turn-around times. • Automation of tests is diffi cult. You’ll need a physical device attached to your build server to run your tests. • The test suite setup is done manually with Java ME unit testing frameworks. Each time you add or remove a test, you have to update your test suite setup. This takes unnecessary time and is error prone.

These issues lead the Java ME developers to the darker side of testing: integration unit testing. Integration unit testing means that you don’t fully isolate your unit un- der test, but rather let it talk to real instances of all or some of its collaborators. There are however some problems with integration unit testing: It catches errors too late in the development process, it’s complex to set up, and it often causes seemingly end- less problems when trying to manage the test data. Unit tests based on mock objects, on the other hand, require a minimum of test data, they are written together with your production code, and they verify that your code actually does what you want it to do. Java SE has elegant solutions to all of the above issues. Why not use them?

Introducing MockME

For Java ME developers, there is an underlying assumption that unit tests need to be run on the target device. This is not true. Consider a Java EE project: An enterprise application may be targeted to run on an application server on other hardware and JVM vendor from our development environment. The unit tests are still run in the development environment. How does it work? The ability to ”write once, run everywhere” is one of the greatest strengths of Java. However, any experienced Java developer knows that this is not completely true. There can be some compatibility issues (e.g. threading), but for unit tests this will not be a problem. We’re only testing a small unit in a controlled setup. Given that the above statement is true, we could run our tests in a Java SE en- vironment and utilize all its excellent tools! This is the basic idea behind MockME; to write unit tests for Java ME applications and run them in Java SE. So how does it work? MockME contains empty objects for CLDC, Mobile Information Device Profi le (MIDP), and other JSRs designed to run in Java SE. Then we use EasyMock to create mock objects dynamically from MockME. Now, some might ask: ”Why not create the mock objects from the classes in a wireless toolkit (WTK)”? Unfor- tunately, these toolkits contain native methods that can’t be mocked. Even if it was possible, EasyMock can’t mock static methods, such as RecordStore.openRecordS tore(String,boolean). MockME solves this by delegating static method invocations to a static delegate instance with a special method name. Let’s have a look at the inner workings:

3

RecordStore.java

public class RecordStore { // Used to delegate static method invocations private static RecordStore _delegate;

// Used to set static delegate, i.e. a mock object public static void setStaticInstance(RecordStore delegate) { _delegate = delegate;

}

// The static method delegates to a non-static equivalent public static RecordStore openRecordStore(String recordStoreName, boolean createIfNecessary)

throws RecordStoreException, RecordStoreFullException, RecordStoreNotFoundException { return _delegate.static_openRecordStore(recordStoreName, createIfNeces- sary); }

  • 4 // This method represent the static method and can be mocked! protected RecordStore static_openRecordStore(String recordStoreName, boolean createIfNecessary)

throws RecordStoreException, RecordStoreFullException, RecordStoreNotFoundException { return null;

}

// ...

}

By creating a mock object and setting the static instance, we fake the static method invocations. No real magic here, but it’s important to make the development of Java ME unit tests much simpler.

MockME In Action

OK, after motivating MockME, let’s look at a small example:

MessageDao.java

public class MessageDao { private static fi nal String RECORD_STORE_NAME = ”MessageDao”;

public void addMessage(Message message) throws RecordStoreException { RecordStore recordStore = null; try { // This static method invocation is normally hard to mock recordStore = RecordStore.openRecordStore(RECORD_STORE_NAME, true); byte[] bytes = encodeMessage(message); if (message.getId() == -1) { message.getId() = recordStore.addRecord(bytes, 0, bytes.length); } else { recordStore.setRecord(message.getId(), bytes, 0, bytes.length);

} } fi nally { if (recordStore != null) { recordStore.closeRecordStore();

}

}

}

private byte[] encodeMessage(Message message) {

// Convert message to bytecode

byte[] bytes =

;

return bytes;

}

}

The class above is an example of a simple Data Access Object (DAO). It allows us to store and access messages from a record store. Now, we’d like to create a unit test for the addMessage() method. It’s especially important to make sure that we always close the record store after use. This should be done even if we get a RecordStore- FullException. Let’s create test methods for both cases:

MessageDaoTest.java

public class MessageDaoTest extends TestCase {

private MessageDao messageDao;

// Object under test

private RecordStore recordStoreMock; // Mock object

// Called by JUnit to initialize the test public void setUp() { // Create mock object for RecordStore and use the special method // provided by MockME to set the static instance recordStoreMock = createMock(RecordStore.class); RecordStore.setStaticInstance(recordStoreMock);

// Create object under test messageDao = new MessageDao();

}

// Called by JUnit to clean up after each test run public void tearDown() { messageDao = null; recordStoreMock = null;

}

public void testAddMessage() throws Exception { // Set up expected behaviour byte[] expectEncMess = ”\u0004from\u0004text”.getBytes(); expect(RecordStore.openRecordStore(”MessageDao”, true)) .andReturn(recordStoreMock); expect(recordStoreMock.addRecord(aryEq(expectEncMess), eq(0), eq(10)))

.andReturn(42);

recordStoreMock.closeRecordStore();

// Put the mock object in test state recordStoreMock.replay();

// Executing the code we want to test Message message = new Message(”from”, ”text”);

messageDao.addMessage(message);

// Verify the behaviour recordStoreMock.verify();

assertEquals(”Wrong message ID”, 42, message.getId());

}

public void testRecordStoreFull() throws Exception { // Set up expected behaviour expect(RecordStore.openRecordStore(”MessageDao”, true)) .andReturn(recordStoreMock); expect(recordStoreMock.addRecord(eq(42), (byte[]) anyObject(), eq(0), eq(10))) .andThrow(new RecordStoreFullException()); recordStoreMock.closeRecordStore();

// Put the mock object in test state recordStoreMock.replay();

// Executing the code we want to test try { Message message = new Message(”from”, ”text”); messageDao.addMessage(message); fail(”Expected exception”); } catch (RecordStoreFullException e) { assert(true); // Ok, expected

}

// Verify the behaviour recordStoreMock.verify();

}

}

Lets look at the setUp() method. First we use EasyMock to create a mocked RecordStore object dynamically, that the MessageDao is going to write its data to. Then we invoke RecordStore.setStaticInstance() to mock all static methods in RecordStore. Static method invocations will be delegated to static_<method name>() equivalents. Next is the test method testAddMessage(). The method starts preparing the en- coded message and continues by setting up the expected behavior of the mock object. You do this in EasyMock by invoking methods directly on the mock object. In a sense, EasyMock ”records” these expected method invocations so it can com- pare, or replay, the expectations with the actual invocations later, when the method under test is called. If a method has a return value, you need to use a (statically imported) expect() method. By using the andReturn() method, we can return a mocked value. Before executing the code we want to test, we invoke replay() to stop the mock object from recording and set it to the replay state. All recorded method invocations will be verifi ed from now on. After executing the code under test, we verify the behavior by calling verify() on all mock objects. Finally, we use a standard JUnit assert() method to assert the message ID. That’s it! The next test method uses the same principle, except that we let EasyMock throw an exception during ad- dRecord() to test that we really close the record store even if we get an exception. A test that could be quite diffi cult if running in a Java ME environment. This is one ex- ample of how to use MockME and EasyMock for unit testing Java ME applications. Once you get the hang of it, it’s easy to unit test your Java ME code in Java SE.

Eyes On The Prize

We’ve now seen how MockME has given us the opportunity to use Java SE tools, such as EasyMock, for Java ME unit tests. But there is so much more, for example code coverage. A code coverage tool is used to detect what parts of your code are executed. This is a great tool for motivating developers to write test code. A code coverage report indicates the percentage of your code that was tested (covered), normally indicated with a green bar of variable length. It’s highly motivating, and fun, to see how the green bar grows. Remember that it is not the number of test cas- es that is important, but what parts of your code are executed by your test cases. You are not limited to the above mentioned tools. You could use any Java SE unit testing tool or testing library. For example, there are libraries that simplify testing of multi-threaded classes, comparing XML documents, and so on. This would not be possible, or at least it would be very hard, if you were to use one of the traditional Java ME testing frameworks.

Conclusion

In this article we have shown how to write true unit tests for Java ME applications. Running the tests in a Java SE environment gives us a lot of advantages and is pos- sible thanks to MockME. So, should we throw away all our Mobile JUnit test code? No. It’s important to distinguish between unit tests and integration unit tests. We have stressed the use of true unit tests in Java ME, something rarely seen in Java ME projects. There is always a need to test parts of your application on the target platform, but in combination with true unit tests, the integration unit tests will be fewer and more focused on integration issues.

References

Magnus Robertsson and Johan Karlsson

Jayway

5

Dependency Injection: http://www.martinfowler.com/articles/injection.html Mock Objects: http://www.connextra.com/aboutUs/mockobjects.pdf MockME: http://www.mockme.org/ JUnit: http://www.junit.org/ EasyMock: http://www.easymock.org/ J2MEUnit: http://j2meunit.sourceforge.net/ JMUnit: http://sourceforge.net/projects/jmunit/ Mobile JUnit: http://developer.sonyericsson.com/site/global/docstools/java/p_java.jsp

Signing MIDlets

W ith today’s functionality and information-crammed mobile devices the threat

of malicious code is a real concern, especially for those of us that like to download and install third party ap- plications. It is essential for the growth of the mobile content market that cus-

  • 6 tomers can feel safe using their appli- cations without worries of stolen data or unexpected billing. The Java ME platform addresses this problem by protection domains, permissions and trusted MIDlet suites. In short, a trusted MIDlet is one whose supplier can be securely identifi ed as trustworthy at the time of installation in contrast to other MIDlet suites where nothing can be guaranteed about the supplier. The trustworthiness is proven by a certifi cate and allows the applica- tion to run in a permissive protection domain where a selected set of poten- tially risky functionalities on the device are made available, no questions asked (or at least not more than once). This relieves the end user from repeatedly having to allow access to the fi le system,

network connections or similar which is often the case otherwise.

Certifi cates and code signing - quick and dirty

Code signing and certifi cates rely on the principles of asymmetric encryption

schemes such as RSA, where each in- dividual has a freely distributed public key and a secret private key. The two keys work in reverse when using the RSA algorithm, so that if one of them is used to transform plain data into an encrypted form then the other one can be used for recovery. So when shipping a piece of software, the vendor creates a signature of the code (or rather a hash value of it) us- ing the private key. The end user in turn calculates a hash value of the received code, applies the public key on the ven- dor’s signature and then compares the two hash values. If the code has not

– the whys and hows

been tampered with they should match exactly, while a match would be very improbable in any other case. So, the end user can verify the integrity of the code but what about the integrity of the vendor, how can one be sure that the public key really belongs to the party claiming it? Certifi cates to the rescue! When buying a mobile phone a number of public keys are already embedded in the device’s fi rmware, these keys be- long to certifi cate authorities (CA) such as Thawte and Verisign and can be as- sumed to be genuine because of the extensive control measures taken at the phone manufacturing plant. An applica- tion vendor can apply for a certifi cate which essentially is a document binding the vendor’s public key to its identity information by means of a CA signa- ture. This certifi cate is then used by the end user’s system to verify the vendor’s public key and identity, relying on the trustworthiness of the CA.

Sounds great, where do I sign up?

Let’s study the case of the fi ctitious soft- ware publisher FooBar Soft. Their best selling fi le system management product is now to be released on the mobile plat- form, and the technicians have made a brilliant job porting it using the fi le sys- tem access API, JSR-75. However, the application turns out to be a real nuisance to use because of the ever-occurring security popups ask- ing the user to allow the software to do this and that. FooBar Soft realizes that signing the application would be a good idea because it allows instant access to the fi le system, no questions asked. As an additional benefi t, the customers would have increased trust in the ap- plication and be even more keen to try it out. The fi rst step is to generate a key- store with a public/private key pair and a certifi cate signing request (CSR) us- ing the Java SDK command line tool keytool. The CSR is then submitted to a CA bundled with information prov- ing the identity of FooBar Soft. After the CA has accepted the request and a

7

8

8 fee has been paid, a signed certifi cate including the public key is made available

fee has been paid, a signed certifi cate including the public key is made available for download or sent to FooBar via mail so that it can be imported to the keystore using keytool again. There is normally no reason to keep the certifi cate secret so it can be transferred using an insecure channel. Essentially, the keystore now holds the private key used for signing and the certifi cate that is shipped with every signed delivery. A MIDlet suite is distributed in two parts; a Java archive (JAR) fi le including all the classes of the application, and a Java archive description (JAD) fi le containing metadata about the suite. Some metadata is also duplicated in a manifest fi le in- cluded in the JAR. For a signed MIDlet suite, the protected API functions that the application uses have to be listed in the JAD and the manifest fi le. In FooBar Soft’s case, this means adding the following line:

MIDlet-Permissions: javax.microedition.io.Connector.fi le.read, javax.microedi- tion.io.Connector.fi le.write

Adding permissions to the JAD fi le is preferably handled by an IDE, as is the sign- ing. The alternative is to use the command line tool jarsigner, found in Java SDK. Two more items need to be added to the JAD during signing. These are the actual data fi elds that the mobile phone will use for verifi cation during installation, i.e. the signature and certifi cate:

MIDlet-Certifi cate: <certifi cate (Base64 encoded)> MIDlet-Jar-RSA-SHA1: <signature (Base64 encoded)>

Signing a suite is a quick operation, which is often included in the build process also during development. It is important that the customer has a root certifi cate on the phone belonging to the CA that issued the application vendor certifi cate, otherwise the installation will fail. This is also the case if the validity period of the certifi cate has run out or on some platforms if the certifi cate has been revoked. In practice there are only a few CAs whose root certifi cates are widespread in mobile devices and one of these CAs is typically the best choice for the developer.

What’s in a signature and what’s not / The other side of the story

8 fee has been paid, a signed certifi cate including the public key is made available

It is important to realize what a certifi cate and signature really says. It does say that the origin of the software can be settled and that the software has not changed since being shipped. If the supplier is well known, this usually implies a high level of confi dence that the software will do what it is supposed to without harming the phone. However, the CA is only responsi- ble for binding keys to identities, and not to guarantee the function of every appli- cation signed using those keys. In essence, the signature does not say that a trusted MIDlet suite is necessarily well-behaved and a healthy dose of cautiousness is al- ways recommended. It is also vital that the private key really is kept private by the vendor, if not then anyone in posses- sion of it could pose as the rightful owner. Also, as much as code signing increases the trust in applications it also has the potential of locking developers out of a platform. The cost of a certifi cate can be an unfeasible expense for a small scale business or hobbyist. Network operators may also limit the number of root certifi -

cates available in their branded devices, possibly only including their own.

Erik Rydgren

Realway

Essential

Java Generics

O nce you get past simple usage of Java Generics and start implementing

generic classes yourself it may seem quite intimidating. It is tricky, so

it is important to remember a few rules.

Subtyping

The Liskov Substitution Principle, the rule that says that subclasses should be sub- stitutable for their base classes, does not apply to generic elements! • Integer is a subtype of Number.

Integer is a subtype of Comparable.

List is a subtype of Collection.

List<Integer> is a subtype of Collection<Integer>.

List<Integer> is not a subtype of List<Number>.

List<Integer> is not a subtype of List<Comparable>.

// Example why generic elements are not proper subtypes:

List<Integer> integers = new LinkedList<Integer>;

List<Number> numbers = integers;

numbers.add(3.14);

// Won’t compile! // Integers cannot contain 3.14

The fact that the Liskovs Substitution Principle does not apply to generics restricts their usefulness; we need wildcards to loosen some of these restrictions.

Wildcards

To loosen the constraint above, wildcards may be used. Wildcards are used with the keywords extends and super. • <? extends Number> means all types that are subclasses of Number are allowed. • <? super Integer> means all types that are superclasses of Integer are al- lowed.

The Get and Put Principle: use extends only when you intend to get values out of a structure. Use super only when you intend to put values into a structure.

The container that you get something out of is guaranteed to contain elements that are instances of the expected class or of a subclass and may be used properly by the recipient of the get.

The container that you put something into is guaranteed to contain at least in- stances of the expected class or a superclass ensuring that the put is valid.

This also implies: don’t use any wildcards when you intend to both get and put values into and out of a structure.

// Extends wildcard violation List<Integer> integers = new LinkedList<Integer>();

List<? extends Number> numbers = integers; numbers.get(i);

numbers.add(3);

// Works fi ne! // Won’t compile!

// Super wildcard violation List<Number> numbers = new LinkedList<Number>();

List<? super Integer> integers = numbers;

integers.add(3);

int i = integers.get(0); Object o = integers.get(0);

// Works fi ne! // Won’t’ compile! // Works fi ne, object is upper bound!

// Copy all elements, subclasses of T, from source to dest which con- tains elements that are superclasses of T.

A real tapir in the wild JayView
A real tapir in the wild
JayView

9

public static <T> void copy(List<? super T> dest, List<? extends T> source) { for (int i = 0; i < source.size(); i++) { dest.set(i, source.get(i));

}

}

In addition to the above principle there are also a few restrictions on wildcards:

Don’t use wildcards when creating instances, specifying generic method calls or ex- tending superclasses:

List<?> integers = new LinkedList<?>(); creation! List<?> integers = Lists.<?>factory();

method!

// Won’t compile, instance

// Won’t compile, generic

  • 10 class AnyList implements List<?> {} // Won’t compile, extend su- perclass! The syntax above is valid if any of the question marks is replaced by a proper class. The generic method is shown here in case the syntax looks unfamiliar. class Lists { static <T> List<T> factory() { return new LinkedList<T>(); }

}

Bounds

Bounds are used to make sure that generic parameters are of a specifi ed subtype.

// The generic parameter of Query must extend (or implement) Entity and Entity must have a getId method! public class Dao<T extends Entity> { T createOrUpdate(T entity) { if (entity.getId() != null) { return update(entity); } else { return create(entity);

}

}

}

// Using Dao // Works fi ne since Person extends Entity! Dao<Person> personDao = new Dao<Person>();

// Wont compile since String does not extend Entity! Dao<String> stringDao = new Dao<String>();

Bounds may also be used in more advanced ways. The example below is a simplifi ed version from java.util.Collections and show a recursive bound. The generic parameter T is also used inside the bound Comparable<T> to make sure that the objects contained in the collection are comparable amongst themselves.

// The method max takes a parameter which must contain elements of a subclass of Comparable. // In addition the Comparable class must be comparable with the declared type public static <T extends Comparable<T>> T max(Collection<T> collection) {

T currentMax = collection.iterator().next(); for (T element: collection) { if (currentMax.compareTo(element) < 0) currentMax = element;

} return currentMax;

}

By far the most diffi cult generic declaration comes from java.lang.Enum and looks

like this Class Enum<E extends Enum<E>> implements Comparable<E>. Like the

above declaration this is a recursive bound but it is even more constrained than the above. The key to understanding this is to know how enums are implemented in Java.

// Declaring an enum enum Tapir {MALAYAN, BRAZILIAN, BAIRD, MOUNTAIN}

// creates a class similar to this! public fi nal class Tapir extends Enum<Tapir> implements Comparable<Tapir> {

private Tapir(String name, int ordinal) {super(name, ordinal)} public static fi nal Tapir MALAYAN = new Tapir(”MALAYAN”, 0); public static fi nal Tapir BRAZILIAN = new Tapir(”BRAZILIAN”, 1); public static fi nal Tapir BAIRD = new Tapir(”BAIRD”, 2); public static fi nal Tapir MOUNTAIN = new Tapir(”MOUNTAIN”, 3); private static fi nal Tapir[] VALUES = {MALAYAN, BRAZILIAN, BAIRD, MOUNTAIN}; public static Tapir[] values() {return VALUES.clone()}

public static Tapir valueOf(String name) { for (Tapir t: VALUES) if t.getName().equals(name) return t; throw new IllegalArumentException();

}

}

As can be seen in the code above E extends Enum<E> maps to Tapir extends

Enum<Tapir> and Comparable<E> maps to Comparable<Tapir>. This makes sure

that enums of one type can only be compared with enums of the same type. The innermost generic parameter, Enum<E extends Enum<E>>, makes the subclass’ type available to the superclass, allowing it to defi ne methods whose parameters and return values are that of the subclass’.

Generic parameters may also have multiple bounds. The signature of the simplifi ed example above actually looks like this:

// Actual signature of max from Java Collections public static <T extends Object & Comparable<? super T>> T max(Collection<? extends T> collection);

When multiple bounds appear the fi rst bound is used for erasure and the reason for the Object in the signature above is that it makes the signature backwardly compat- ible. The reason for the super and the extends are the same as above to make the method more generic.

The method takes a collection of Ts or a collection of Ts subclasses (Collection<? ex- tends T>) where T or one of Ts superclasses implements Comparable with erasure Object (<T extends Object & Comparable<? super T>>).

Erasure

Java Generics is implemented by erasure. This means that the generic information is removed when the class is compiled:

The erasure of List<Integer>, List<String>, List<List<Integer>> is List. The erasure of Comparable<? super T> is Comparable.

• The erasure of Object & Comparable is the leftmost, Object.

Another thing to be aware of is that the implementation of generics with erasure forces the compiler to insert additional methods into the class fi les.

// Additional methods are compiled into generic classes public interface Foo<T> { void foo(T param);

}

public class Bar implements Foo<Bar> {

// This method will appear twice once with Object as parameter and once with Bar. public void foo(Bar param) {}

public static void main(String[] args) { for (Method m : Bar.class.getMethods()) if (m.getName().startsWith(”foo”)) System.out.println(m.toGenericString());

}

}

$ java Bar public void Bar.foo(Bar) public volatile void Bar.foo(java.lang.Object)

This can trip you up if you try to overload a method to accept Object as a parameter too. It has never been a good idea to overload with Object as well as a subclass of Object but now it will not even compile:

Error:line (6)name clash: foo(java.lang.Object) in Bar and foo(T) in Foo<Bar> have the same erasure, yet neither overrides the other

Compatibility

All in all, the implementation of generics in Java is a wonderful example of crafts- manship. The solution is binary compatible both backwardly and forwardly, allowing new code to use old libraries as well as allowing old code to use new libraries. I do, however, wish that they had skipped the compatibility and made generic classes aware of what they are at runtime.

11

If you want to know more about generics I highly recommend the book Java Gener- ics by Philip Wadler and Maurice Naftalin.

Anders Janmyr

Ortelius AB

12

Problems with object creation

12 Problems with object creation E very programming language has tricky details that you need to

E very programming language has tricky details that you need to be aware

of. This article will look at several such issues in Java related to the Java

compiler. Test your Java skills and see if you know the answer!

The problem

A while ago a colleague of mine discovered a weird behavior when writing some integration tests. He fi rst noticed that the test worked in JDeveloper, but not in Eclipse. Then he noticed that he could make a small change to the test setup and the test would pass. Another small change and it would fail again. He got some help and managed to reduce the problem to the following:

Original code

import junit.framework.TestCase;

public class ObjectCreationTest extends TestCase { class MyObject { Object temp;

public MyObject() { set();

} void set() { } Object get() { return temp;

}

}

public void testWithNewString() throws Throwable {

fi nal String value = new String(”Value”); assertEquals(”Value”, value);

MyObject tested = new MyObject() { void set() { temp = value;

};

}

String result = (String) tested.get(); assertEquals(”Value”, result);

}

}

This test fails in some environments and works in others. When the test fails it is because result is null. How can this be? But it gets even stranger: By changing how the value is initialized the test always pass:

Using a String constant

public void testWithConstant() throws Throwable {

fi nal String value = ”Value”; assertEquals(”Value”, value);

MyObject tested = new MyObject() { void set() { temp = value;

};

}

String result = (String) tested.get(); assertEquals(”Value”, result);

}

This test works for all environments. Why does the initialization of value affect result? Very strange indeed! Finally, by changing the type of the value we are back to the situation where the test case fails in some environments, but not all.

Using Object instead of String

public void testWithObject() throws Throwable {

fi nal Object value = ”Value”; assertEquals(”Value”, value);

MyObject tested = new MyObject() { void set() { temp = value;

};

}

String result = (String) tested.get(); assertEquals(”Value”, result);

}

Challenge

Here are some challenges for you! Try to solve them before reading the solution! • Explain the difference between the middle test case and the other two (easy). • Explain why the initial test case fails in some environments and works in others (hard). • Find the bad design that might cause problems (and indeed does in this case!).

I can give you a hint. The differences are both related to the Java compiler. Don’t cheat! You should at least be able to answer the irst question before reading on.

Solution

There are several things going on here. I’ll start by explaining what is going on when we change the variable named ”value” and then I’ll look more closely at anonymous inner classes. Finally I will describe the problem with the design.

Constant variables

Here are the three cases again:

final String value1 = new String(”Value”); final String value2 = ”Value”; final Object value3 = ”Value”;

According to the Java Language Specifi cation (JLS) ”We call a variable, of primitive type or type String, that is fi nal and initialized with a compile-time constant expres- sion (§15.28) a constant variable”. Lets go through each variable:

• ”value1” is not a constant since it does not have a compile-time constant expres- sion! That is, the compiler does not know what the String constructor is doing and therefore assumes that this needs to be evaluated at runtime. • ”value2” is a constant variable since it is clearly of type String and also initialized with a compile-time constant value. • ”value3” is not a constant since it is not a primitive type or a String. Yes, the object it refers to is a String, but the variable itself is an Object.

This means that only the middle case is a constant. When the compiler spots a constant it automatically replaces all references with the constant value, ie in the compiled class a constant variable is never referenced! The result is that the next problem I will describe does not occur and the test case works.

Anonymous inner classes

As you know an inner class is able to access not only fi elds in the creating instance but also fi nal variables in the method that creates the class. Did you ever wonder how this magic is done? Maybe you have also noticed that it is not possible for an anonymous class to have a constructor. When you create an anonymous inner class the compiler actually creates the constructor for you. Let’s decompile the original version and see what is going on.

Decompiled the original version (using JAD and javap)

fi nal class ObjectCreationTest$1 extends MyObject { fi nal ObjectCreationTest this$0; private fi nal Object val$value;

ObjectCreationTest$1(ObjectCreationTest that, Object obj) this$0 = that; val$value = obj; super();

} void set() { temp = val$value; }

} public void testWithNewString() throws Throwable{ fi nal Object value = new String(”value”); MyObject tested = new ObjectCreationTest$1(this, value); Object result = tested.get(); assertEquals(value, result);

}

13

Now it is easy to see that there is no magic going on! The compiler has simply added a constructor and two instance variables. The set method is not using the fi nal vari- able from the method, instead it is simply using its own reference to the value. Well, this is all very nice, but why did the test case fail in some environments? Notice how the generated constructor fi rst initializes the members and then calls

JayView

super. This is normally not allowed in Java, but consider what would happen if the compiler didn’t work this way. MyObject constructor would be called, which then

calls set and tries to use val$value which has not been initialized yet. Because of a bug in the compiler this is exactly what happened in Java compilers before release

1.4.

OK, so if this problem is fi xed in 1.4 and later, why should you care? Take a look

at the next problem!

Bad design

The problem with the design is that the base class MyObject is calling methods in the anonymous subclass before the subclass instance variables have been created. Unfor- tunately this is not only a problem with anonymous classes, but a problem in general. Take a look at the following test case and see if you know what will happen.

public class BaseCallingSubTest extends TestCase {

  • 14 class Base { Base() {

doStuff();

} void doStuff() { }

} class Sub extends Base { private fi nal int fi nalField = 5; private int normalField = 5; void doStuff() { assertEquals(5, fi nalField); assertEquals(5, normalField);

}

}

public void testSub() { Sub s = new Sub();

}

}

The test case will fail at the second assertEquals. Why? Two things are happening.

First of all ”fi nalField” is a constant variable and the compiler automatically uses 5 instead of referencing the variable, see above. Secondly, the constructor for Base is called before the fi elds of Sub are initialized! Things are actually happening in this order:

  • 1. The fi elds in Base are initialized

  • 2. Base constructor is run

  • 3. The fi elds in Sub are initialized

  • 4. Sub constructor is run

This is not a bug! This is exactly according to JLS ”§12.5 Creation of New Class Instances”. The following is perhaps even more surprising:

public class BaseCallingSub2Test extends TestCase {

class Base { Base() {

doStuff();

} void doStuff() { }

} class Sub extends Base { private int normalField = 5; void doStuff() { normalField = 7;

}

}

public void testBase() { Sub s = new Sub(); assertEquals(7, s.normalField);

}

}

This test case also fails because after Base and doStuff have been called, Sub is ini- tialized and the normalField is assigned to 5. This is possible to solve by not initial- izing normalField.

class Sub extends Base { private int normalField; void doStuff() { normalField = 7;

}

}

This works as expected. However, try to avoid constructions like this as it is very easy to forget that the fi elds might not be initialized yet. There are actually more problems with this construction as it might affect thread safety. For instance if the subclass starts a thread in the overridden method this thread might be given access to the uninitialized object. Brian Goetz (author of ”Java Concurrency in Practice”) goes as far as calling this ”not properly constructed”.

Lessons learned

• Make sure you understand constant variables and how they are used by the com- piler Use a recent version of Java compiler. Sun is constantly adding improvements and fi xing bugs. • Avoid calling non fi nal methods from the constructor. If you have to, be aware that the object might not be initialized yet. • Do not perform unnecessary initialization of fi elds.

Jan Kronquist

Jayway

Neo – a netbase

– a netbase N eo is a network-oriented database for semi-structured information. Too complicated, let

N eo is a network-oriented database for semi-structured information. Too complicated, let us try again. Neo handles data in networks – nodes, relationships and properties – instead of tables. This means

entirely new solutions for data that is diffi cult to handle in static tables. It could mean we can go agile all the way into the persistence layer.

The relational database represents one of the most important developments in the history of computer science. Upon its arrival some 30 years ago, it revolutionized the way the industry views data management and today it is practically ubiquitous.

In fact, it is so taken for granted that we, as an industry, have stopped thinking. Could there be a better way to represent and store our data? In some cases the answer is – yes, absolutely. The relational database is showing its age. Some telltale signs:

• The mismatch between relational data and object oriented programming. • Schema evolution – updating your data model when the domain model changes – is just so much manual labor. • Semi-structured or network-oriented data is diffi cult to handle.

The Neo Database

Neo is a database that is built with a different philosophy. Where the relational da- tabase squeezes all data into static tables, Neo uses a fl exible and dynamic network model. This model allows data to evolve more naturally as business requirements ”

change. There’s no need for “alter table

on your production databases after you

... introduce a new version of the business layer and no need to rewire and migrate your O/R mapping confi gurations. The network will evolve along with your busi- ness logic. This spells agility. Neo is an embedded persistence engine, which means it’s a small, lightweight and non-intrusive Java library that is easy to include in your development environment. It has been designed for performance and scalability and has been proven to handle large networks of data (100+ millions of nodes, relationships and properties). Neo is a newly founded open source project, but the software is robust. It has been in commercial production in a highly demanding 24/7 environment for al- most four years and has full support for enterprise-grade features such as distributed ACID transactions, confi gurable isolation levels and full transaction recovery. But so much for sweet talk, let’s cut to some code!

Model and Code

Representation

15

In the Neo model, everything is represented by nodes, relationships and properties. A relationship connects two nodes and has a well-defi ned, mandatory type. Prop- erties are key-value pairs that are attached to both nodes and relationships. When you combine nodes, relationships between them and properties on both nodes and relationships they form a node space – a coherent network representing your busi- ness domain data. This may sound fancy, but it’s all very intuitive. Here is how a simple social network might be modeled:

Figure 1: An example of a social network from a somewhat famous movie. Note the different
Figure 1: An example of a social network from a somewhat famous movie. Note the different
Figure 1: An example of a social network from a somewhat famous movie. Note the different
Figure 1: An example of a social network from a somewhat famous movie. Note the different
Figure 1: An example of a social network from a somewhat famous movie. Note the different
Figure 1: An example of a social network from a somewhat famous movie. Note the different
Figure 1: An example of a social network from a somewhat famous movie. Note the different

Figure 1: An example of a social network from a somewhat famous movie. Note the different type on the relation between Agent Smith and his creator The Architect.

16

Note how all nodes have integer identifi ers and how all relationships have a type (KNOWS or CODED_BY). In this example, all nodes have a “name” property. But some nodes have other properties, for example, an “age” property (node 1) or a “last name” property (node 3). There’s no overall schema that forces all nodes to look the same. This allows Neo to capture so-called semi-structured information:

information that has a small amount of mandatory attributes but many optional at- tributes. Furthermore, the relationships have properties as well. In this example, all relationships have an “age” property to describe how long two people have known each other and some relationships have a “disclosure” property to describe whether the acquaintance is secret.

Working with nodes and relationships is easy. The basic operations are as follows:

Figure 1: An example of a social network from a somewhat famous movie. Note the different
Figure 1: An example of a social network from a somewhat famous movie. Note the different

This is an intuitive representation of a network and probably similar to many other implementations that want to represent a network of data in an object-oriented language. It’s worth noting, however, that relationships in this model are full-blown objects and not just implicit associations between nodes. If you have another look at the social network example, you’ll see that there’s more information in the relationships between nodes than in the nodes themselves. The value of a network is in the con- nections between the nodes and Neo’s model captures that.

Creating a Node Space

And now, fi nally some code. Here’s how we would create the Matrix social network from fi gure 1:

Transaction tx = Transaction.begin();

EmbeddedNeo neo =

...

// Get factory

// Create Thomas ’Neo’ Anderson Node mrAnderson = neo.createNode(); mrAnderson.setProperty( ”name”, ”Thomas Anderson” ); mrAnderson.setProperty( ”age”, 29 );

// Create Morpheus Node morpheus = neo.createNode(); morpheus.setProperty( ”name”, ”Morpheus” ); morpheus.setProperty( ”rank”, ”Captain” ); morpheus.setProperty( ”occupation”, ”Total bad ass” );

// Create a relationship representing that they know each other mrAnderson.createRelationshipTo( morpheus, MatrixRelationshipTypes.KNOWS );

// Create Trinity, Cypher, Agent Smith, Architect similarly ...

tx.commit();

As you can see in the code above: It is rather easy to construct the node space for our Matrix example. And, of course, our network is made persistent once we commit.

Traversing a Node Space

Now that we know how to represent our domain model in the node space, how do we get information out of it? Unlike a relational database, Neo does not support a declarative query language. Instead, Neo provides an object-oriented traverser frame- work that allows us to express complex queries in plain Java. Working with the traverser framework is very straight-forward. The core abstrac- tion is, unsurprisingly, the Traverser interface. A Traverser is a Java Iterable that encapsulates a “query” – i.e. a traversal on the node space such as “give me all Mor- pheus’ friends and his friends’ friends” or “does Trinity know someone who is acquainted with an agent?”. The most complex part of working with a Traverser is instantiating it. Here’s an example of how we would create a Traverser that will return all the (transitive) friends of the “Thomas Anderson” node of the example above:

// Instantiate a traverser that returns all mrAnderson’s friends Traverser friendsTraverser = mrAnderson.traverse( Traverser.Order.BREADTH_FIRST, StopEvaluator.END_OF_NETWORK, ReturnableEvaluator.ALL_BUT_START_NODE, MatrixRelationshipTypes.KNOWS, Direction.OUTGOING );

Here we can see that traversers are created by invoking the traverse( method on a start node with a number of parameters. The parameters control the traver- sal and in this example they tell Neo to traverse the network breadth-fi rst (rather than depth-fi rst), to traverse until it has covered all reachable nodes in the network

...

)

(StopEvaluator.END_OF_NETWORK), to return all nodes except the fi rst (Returna-

bleEvaluator.ALL_BUT_START_NODE), , and to traverse all OUTGOING relation- ships of type KNOWS. How would we go about if we wanted to list the output of this traversal? After we’ve created a Traverser, working with it is as easy as working with any Java Iter-

able:

// Traverse the node space and print out the result for ( Node friend : friendsTraverser ) {

System.out.println( friend.getProperty( “name” ) + “ at depth “ + friendsTraverser.currentPosition().getDepth() );

}

Running the traversal above on the Matrix example would yield the following out- put:

$ bin/run-neo-example Morpheus at depth 1 Trinity at depth 1 Cypher at depth 2 Agent Smith at depth 3 $

As you can see, the Traverser has started at the “Thomas Anderson” node and run through the entire network along the KNOWS relationship type, breadth fi rst, and returned all nodes except the fi rst one. “The Architect” is missing from this output since the relationship connecting him is of a different type, CODED_BY. This is a small, contrived example. But the code would work equally well on a network with hundreds of millions of nodes, relationships and properties. Now, let’s look at a more complex traversal. Going with our example, suppose that we wanted to fi nd all “hackers of the Matrix,” where we defi ne a hacker of the Matrix as any node that you reach through a CODED_BY relationship. How would we create a Traverser that gives us those nodes? First off, we want to traverse both our relationship types (KNOWS and COD- ED_BY). Secondly, we want to traverse until the end of the network and lastly, we want to return only nodes which we came to through a CODED_BY relationship. Here’s the code:

// Instantiate a traverser that returns all hackers of the Matrix Traverser hackerTraverser = mrAnderson.traverse( Traverser.Order.BREADTH_FIRST, StopEvaluator.END_OF_NETWORK, new ReturnableEvaluator() {

public boolean isReturnableNode( TraversalPosition pos ) {

return pos.getLastRelationshipTraversed(). isType( MatrixRelationshipTypes.CODED_BY );

}

}, MatrixRelationshipTypes.CODED_BY, Direction.OUTGOING, MatrixRelationshipTypes.KNOWS, Direction.OUTGOING );

Now it’s getting interesting! The ReturnableEvaluator.ALL_BUT_START_NODE con-

stant from the previous example was actually a convenience implementation of the ReturnableEvaluator interface. This interface contains a single method and you can supply a custom implementation of it to the traverser framework. It turns out that this is a simple but powerful way to express complex queries. Setting aside the anonymous inner class cruft surrounding the code in bold, we basically pass in a snippet of code that checks whether we traversed a relationship of type CODED_BY to get to the current node. If this statement is evaluated to “true” then the current node will be included in the set of nodes that is returned from the traverser. When executed with a simple print loop, the above code prints the following:

$ bin/run-neo-example The Architect $

StopEvaluators work the same way. In our experience, writing custom evaluators is very easy. Even the most advanced applications we have developed with Neo – applications that traverse extremely large and complex networks – are based on evaluators that are rarely more than a few lines of code.

Conclusion

Neo is not a silver bullet and some areas needs to improve, for instance tools, stand- ardizing the model and a query language. However, if your data is naturally ordered in a network or is semi-structured or you just need to go truly agile, give the Neo database a run for your money. We hope you fi nd it, as we do, to be an elegant and fl exible alternative that is both robust and fast.

Links Neo specifi cation www.neo4j.org
Links
Neo specifi cation
www.neo4j.org

Emil Eifrém

Neo Persistence

Björn Granvik

Jayway

17

18

  • I am in search of an empty room at the Øredev conference. Normally this is an easy task, but I’ve got

Martin Fowler on my tail. My mind is still blank. What on earth can I ask him that he hasn’t already written himself?

Finally, an empty room, well almost. Another speaker, Erik Dörnenburg, is sitting half way into his screen and mut- ters. - What’s up, I ask. - I’ve updated my machine and my demo doesn’t work. I’ve got 45 minutes until the presentation. We sat down next to him. Do not disturb a developer while he’s coding ... So I got a man who has coined phrases like Dependency Injection and POJO in front of me. What next? Martin is easily recognizable both in accent and appear- ance, a frequent and brilliant speaker. He has an excellent web site, www. martinfowler.com, which contains loads about his work. Articles and references abound. That is when it suddenly hits me - who is he as a programmer and person?

When was the last time you coded?

Well, I do code my own website. But it’s been a while since I had any paying cus- tomers. I’ve been pairing quite recently though. A real delivery? That was some time ago. I’m actually afraid to lose con- tact with code, but I have smart people around me.

But what makes you tick?

  • I enjoy trying to fi gure out new tech-

niques - to organize knowledge. I see myself as a conduit [ledning] for trans- ferring knowledge, to process what is out there and make some kind of structure out of it. Brian Foot actually described me as an ”intellectual jackal with a good taste in carrion” [intellektuell schackal med god smak för kadaver]. I look around for interesting stuff and try to make sense of it. The ”Refactoring” is a good exam- ple. I fi gured out how to describe it and wrote a book that came out when it could make a difference and move the area forward. I also enjoy writing a lot, that’s a big thing. I’m better now at speaking, but that’s not what makes me tick.

You’ve written quite a few books - how do they compare?

Out of the fi ve, ”Uml Distilled” sold more copies than the others put togeth- er. Usually you can’t make a living out of your books, I guess I could though. All of the books had their good sides,

but I would have to say that it was fun to write with Kent Beck [red: wrote ”Extreme Programming”, created JUnit etc]. We were in tune and could support each other through the dull bits.

  • I would have to say though that I’m

most proud of ”Refactoring”. It’s an important technique and didn’t get the attention it should have received – the book helped.

How did you start out?

  • I was an independent consultant for

many years. Giving talks was a good way

of getting jobs. Articles same thing - it

got my name known. Also, I write something because I don’t understand a certain area or tech- nology. It’s a good way to learn.

Erik is now on the phone with Califor-

nia. We calculate that time is roughly 6:30 there – in the morning.

Then what? How come you started working for Thoughtworks (TW)?

I’ve been there for six years and done a lot of consulting. I never wanted to work for a company, but there was something about TW that made me interested. Get the work done and tons of bright people. But more importantly is that it is a sort of social experiment. A notion that good people makes a difference. I hope we can affect IT, which is a dif- fi cult and skilled exercise at best.

What is the most diffi cult part of being a celebrity?

I’m not an extrovert person. I’m not good at the ”person to person”. I get

Looking forward, what’s next?

Oh, there is tons of stuff to write about.

The design patterns area for instance. I’m also interested in DSL [domain spe- cifi c languages] and agile development. But in agile there are too many writers and I don’t like competition. There are too many smart people in agile develop- ment. My strategy is to look for topics that no one has written about. Basically I don’t foretell the future.

What are your top three pieces of ad- vice to a programmer?

My fi rst advice must be to learn to col- laborate with the user or purchaser. The really good ideas usually come from them. You don’t have to be an expert to do this. This I found to be a good general advice. Secondly, it would be ”continuous learning”. It’s like running up a down- wards-moving escalator – you have to keep running. The third one is diffi cult…

Interview

Martin Fowler

– man in the know

emails with questions like ”I got a prob- lem on…what is the magic trick”. They worked for months on it and I can only point to a book. That clearly wasn’t an answer they liked. It’s frustrating. However, celebrity is also a nice thing

– it opens a few doors. I can email people like Rod Johnson [red CEO of Interface 21 that created the Spring framework] if I have a question about something. And he will answer. People tend to think I’m an ingenious programmer. I’m not. I’m pretty good, but not necessarily that great.

Erik suddenly spits out:

- F---! ... ok the demo will be shorter.
- F---!
...
ok
the demo will be shorter.

”Buy lots of books by good authors” would be it.

Erik suddenly releases a big:

- Yes!

I saw Erik’s demo some twenty min- utes later – it was really good. As for Martin, our discussions con- tinued well into the debate panel and beyond. He would frequently forget his back pain and sip into some extra en- ergy pack. I wonder how he did that.

Björn Granvik

Jayway

got my name known. Also, I write something because I don’t understand a certain area or

19

JayView

Java News Terracotta goes open source Terracotta clusters objects at the JVM level without the need
Java News Terracotta goes open source Terracotta clusters objects at the JVM level without the need

Java News

Terracotta goes open source

Terracotta clusters objects at the JVM level without the need of code changes or serialization. For instance, this will enable open source appli- cation servers to use Terracotta for clustering. Also Terracotta ships with examples showing how to cluster a Spring application.

http://tinyurl.com/yfq4rf

Eclipse Mylar 1.0 released

Mylar extends Eclipse user interface with the ability to focus on a single

task. Mylar removes all the clutter and keeps only those things related to the current task, for example a Bugzilla bug.

http://www.eclipse.org/mylar/doc/release-1.0.php

TopLink open sourced

At EclipseCon Oracle announced it will contribute TopLink to Eclipse.

TopLink is a persistence framework for databases, XML and more and im- plements standards such as JPA, JAXB and SDO (next release).

http://tinyurl.com/2nfvwh

Atlassian Bamboo 1.0 released

Bamboo is a commercial product for continuous integration that builds,

tests and generates reports. Besides simple web confi guration, Bamboo also generates statistics and graphs which enables you to see trends in your build process.

http://www.atlassian.com/software/bamboo/

Java News Terracotta goes open source Terracotta clusters objects at the JVM level without the need
Java News Terracotta goes open source Terracotta clusters objects at the JVM level without the need
Java News Terracotta goes open source Terracotta clusters objects at the JVM level without the need

Jaway

Malmö: Hans Michelsensgatan 9 , 211 20 Malmö, +46 40 12 72 83, Sweden København: Fruebjergvej 3, 2100 København Ø, +45 3917 9691, Denmark Helsingborg: Norra Storgatan 8, 252 20 Helsingborg , +46 42 37 35 22, Sweden Malaysia; Sdn. Bhd. B-7-6, Megan Avenue 1, 189 Jalan Tun Razak, 50400 Kuala Lumpur, Malaysia

info@jayway.se

|

www.jayway.se