Report on

Programming of Distributed Applications

Laboratory no.1 - Message Broker
Student: Victor urcanu
Teacher: Dumitru Ciorb
October 23, 2015

Table of Contents
1 Description of the problem
1.1 Messaging . . . . . . . . . . . . . . . . . . . . . . . . . .
1.2 Broking . . . . . . . . . . . . . . . . . . . . . . . . . . .
1.3 Decoupling . . . . . . . . . . . . . . . . . . . . . . . . .
1.4 Developing directions . . . . . . . . . . . . . . . . . . . .
1.4.1 Message Translator (Messaging Systems) . . . . .
1.4.2 Publish-Subscribe Channel (Messaging Channels).
Subscriber . . . . . . . . . . . . . . . . . . . . . .
1.4.3 Content-Based Router (Message Routing) . . . .

3
. . .
3
. . .
4
. . .
5
. . .
6
. . .
6
Durable
. . .
7
. . .
8

2 Description of personal solution

2.1 Steps of elaboration of the application
2.2 Functionalities . . . . . . . . . . . . .
2.3 Use cases . . . . . . . . . . . . . . . .
2.3.1 Scenarios . . . . . . . . . . . .
2.4 Design and application architecture . .
2.4.1 Class Diagram . . . . . . . . .
2.4.2 Component Diagram . . . . . .

.
.
.
.
.
.
.

.
.
.
.
.
.
.

.
.
.
.
.
.
.

.
.
.
.
.
.
.

.
.
.
.
.
.
.

.
.
.
.
.
.
.

.
.
.
.
.
.
.

.
.
.
.
.
.
.

.
.
.
.
.
.
.

.
.
.
.
.
.
.

.
.
.
.
.
.
.

.
.
.
.
.
.
.

.
.
.
.
.
.
.

9
9
9
11
11
13
13
14

3 Code

14

4 Conslusion

18

Description of the problem

Task : Message broker integration which will allow an asynchronous communication between the distributed components of the system.

1.1

Messaging

Messaging is a technology that enables high-speed, asynchronous, programto-program communication with reliable delivery. Programs communicate by
sending packets of data called messages to each other. Channels, also known
as queues, are logical pathways that connect the programs and convey messages. A channel behaves like a collection or array of messages, but one that
is magically shared across multiple computers and can be used concurrently
by multiple applications. A sender or producer is a program that sends a
message by writing the message to a channel. A receiver or consumer is a
program that receives a message by reading (and deleting) it from a channel.
The message itself is simply some sort of data structuresuch as a string,
a byte array, a record, or an object. It can be interpreted simply as data,
as the description of a command to be invoked on the receiver, or as the
description of an event that occurred in the sender. A message actually contains two parts, a header and a body. The header contains meta-information
about the messagewho sent it, where its going, etc.; this information is
used by the messaging system and is mostly (but not always) ignored by the
applications using the messages. The body contains the data being transmitted and is ignored by the messaging system. In conversation, when an
application developer who is using messaging talks about a message, hes
usually referring to the data in the body of the message.[1, p. 13]
Messaging capabilities are typically provided by a separate software system called a messaging system or message-oriented middleware (MOM). A
messaging system manages messaging the way a database system manages
data persistence. Just as an administrator must populate the database with
the schema for an applications data, an administrator must configure the
messaging system with the channels that define the paths of communication between the applications. The messaging system then coordinates and
manages the sending and receiving of messages. The primary purpose of a
database is to make sure each data record is safely persisted, and likewise
the main task of a messaging system is to move messages from the senders
computer to the receivers computer in a reliable fashion.
The reason a messaging system is needed to move messages from one
computer to another is that computers and the networks that connect them
3

are inherently unreliable. Just because one application is ready to send a

communication does not mean that the other application is ready to receive
it. Even if both applications are ready, the network may not be working, or
may fail to transmit the data properly. A messaging system overcomes these
limitations by repeatedly trying to transmit the message until it succeeds.
Under ideal circumstances, the message is transmitted successfully on the
first try, but circumstances are often not ideal. [1, p. 14]
In essence, a message is transmitted in five steps:
1. Create - The sender creates the message and populates it with data.
2. Send - The sender adds the message to a channel.
3. Deliver - The messaging system moves the message from the senders
computer to the receivers computer, making it available to the receiver.
4. Receive - The receiver reads the message from the channel.
5. Process - The receiver extracts the data from the message.

1.2

Broking

A Message Broker takes care of all connections between the applications in an organisation. One connection is created between each system
and Message Broker, and messages transported via this bi-directional connection can be translated, filtered, archived, modified and routed to their
final destination(s).
The transport of messages can be based on one of many transport protocols supported by the broker (e.g. TCP/IP, files). The Message Broker takes
care of translations between the data formats used by the various systems.
All commonly used message formats are supported in the base product (e.g.
HL7, EDIFACT, XML, delimited formats, SAP/HCM, flat records).
The use of a Message Broker offers a scalable platform that has access to
data sent by many applications in the organisation, it improves the reliability
and accessibility of data and it concentrates all available message flows in one
point. Because of the fact that the message broker processes all messages,
a copy of the data contained in the messages can easily be forwarded to
analytical, management database or other applications. Creating a focal
point for data thus adds to the quality of care, the integration of data, and
the central management of the connections.
4

By using Message Broker as a strategic solution for all integration requirements the maintenance costs of point-to-point connections are significantly lowered and new connections can be created at little costs using the
organisations own resources. [3]

1.3

Decoupling

Loose Coupling There is considerable interest in loose coupling in distributed systems, particularly in the web services community. The terminology is often ill-defined and imprecise, though. In the context of web services,
loose coupling refers to minimizing the dependencies between services in order to have a flexible underlying architecture (reducing the risk that a change
in one service will have a knock-on effect on other services). This is partially
supported by the intended independence of web services with the subsequent
intention to produce combinations of web services as discussed above. Loose
coupling is, however, further enhanced by a number of additional features:[4,
p. 385]
Programming with interfaces provides one level of loose coupling by
separating the interface from its implementation (and also supports important areas of heterogeneity, for example in the choice of programming language and platform used). Programming with interfaces is
adopted by most distributed systems paradigms including distributed
objects and components as well as web services.
There is a trend towards simple, generic interfaces in distributed systems and this is exemplified by the minimal interface offered by the
World Wide Web and the REST approach in web services. This approach contributes to loose coupling by reducing dependency on specific operation names. One consequence of this is that data becomes
more important than operation, with the semantics of interoperation
often held in the data (for example, the associated XML document
definition in web services); this data-oriented view is discussed further
in the context of mobile systems.
As mentioned above, web services can be used with a variety of communication paradigms, including request-reply communication, asynchronous messaging or indeed indirect communication paradigms. The
level of coupling is directly affected by this choice. For example, in
request-reply communication, the two parties are intrinsically coupled;

asynchronous messaging offers a degree of decoupling, whereas indirect

communication also offers time and space uncoupling. [4, p. 386]

1.4
1.4.1

Developing directions
Message Translator (Messaging Systems)

In many cases, enterprise integration solutions route messages between existing applications such as legacy systems, packaged applications, homegrown
custom applications, or applications operated by external partners. Each of
these applications is usually built around a proprietary data model. Each
application may have a slightly different notion of the Customer entity , the
attributes that define a Customer and which other entities a Customer is related to. For example, the accounting system may be more interested in the
customers tax payer ID numbers while the customer-relationship management (CRM) system stores phone numbers and addresses. The applications
underlying data model usually drives the design of the physical database
schema, an interface file format or a programming interface (API) those
entities that an integration solution has to interface with. As a result, the
applications expect to receive messages that mimic the applications internal
data format (see Figure 1).

Figure 1: Message Translator.

In addition to the proprietary data models and data formats incorporated in the various applications, integration solutions often times interact
with standardized data formats that seek to be independent from specific
applications. There are a number of consortia and standards bodies that define these protocols, such as RosettaNet, ebXML, OAGIS and many other,
industry specific consortia. In many cases, the integration solution needs to
be able to communicate with external parties using the official data formats
while the internal systems are based on proprietary formats. [1, p. 96]

1.4.2

Publish-Subscribe Channel (Messaging Channels). Durable

Subscriber

A durable subscription saves messages for an inactive subscriber and deliveres

these saved messages when the subscriber reconnects. In this way, a subscriber will not loose any messages even though it disconnected. A durable
subscription has no effect on the behavior of the subscriber or the messaging
system while the subscriber is active (e.g., connected). A connected subscriber acts the same whether its subscription is durable or non-durable.
The difference is in how the messaging system behaves when the subscriber
is disconnected.
A Durable Subscriber is simply a subscriber on a Publish-Subscribe
Channel. However, when the subscriber disconnects from the messaging system, it becomes inactive and the messaging system will save any messages
published on its channel until it becomes active again. Meanwhile, other
subscribers to the same channel may not be durable; theyre non-durable
subscribers.

Figure 2: Publish-Subscribe Channel.

To be a subscriber, the Durable Subscriber must establish its subscription to the channel. Once it has, when it closes its connection, it becomes
inactive. While the subscriber is inactive, the publisher publishes a message. If the subscriber were non-durable, it would miss this message; but
because it is durable, the messaging system saves this message for this subscriber. When the subscriber resubscribes, becoming active once more, the
messaging system delivers the queued message (and any others saved for this
subscriber). The subscriber receives the message and process it (perhaps
7

delegating the message to the application). Once the subscriber is through

processing messages, if it does not wish to receive any more messages, it
closes its connection, becoming inactive again. Since it does not want the
messaging system to save messages for it anymore, it also unsubscribes.[1,
p. 467]
1.4.3

Content-Based Router (Message Routing)

The Content-Based Router examines the message content and routes the
message onto a different channel based on data contained in the message.
The routing can be based on a number of criteria such as existence of fields,
specific field values etc. When implementing a Content-Based Router, special
caution should be taken to make the routing function easy to maintain as the
router can become a point of frequent maintenance. In more sophisticated
integration scenarios, the Content-Based Router can take on the form of a
configurable rules engine that computes the destination channel based on a
set of configurable rules (see Figure 3).

Figure 3: Content Based Routing.

2
2.1

Description of personal solution

Steps of elaboration of the application

In order to decouple the application, there must be created a common interface for the reading/writing files and for sending and receiving messages
through the communication network. This interface is IAsyncIO. This interface must be implemented by the FileIO and NetworkIO classes. These
classes will implement all the logic behind reading, writing from files or for
sending, receiving through the network. Thus, the calling classes must not
be aware of the implementation.
For the implementation I have chosen the Java 1.8 programming language and Java Development Kit 8 (link for JDK 8).
One of the ways to realize the decoupling in Java is to move the specific
component into a separate jar library and to include it at the compilation
of the sourse code. Therefore, our FileIO and NetworkIO, altogether with
IAsyncIO is moved to a separate jar library.
For realizing the Object Oriented Programming approach, the Message,
Parameter and other helping entities were also moved toa diferent package
and compiled in a separate library. The purpose of the enumerated entities
will be specified later on.
Other components of our system are the App and Broker classes. These
are the core classes of our application which perform the logic and provide
the systems main functionalites. Each of these classes is a standalone entity.

2.2

Functionalities

The main purpose of the application is to realize the sending of messages

through a network using UDP or TCP protocol. For this implementation was
chosen the UDP protocol, athough the message delivery is not quaranteed.
The message delivery confirmation is performed by sending a confirmation
message back to the sender entity from the receiving entity.
Another functionality is to read a file from the local storage, load its
content into the body of the message process it accordingly inside the system.
It was chosen to read and interpret the XML file format.
XML is a file extension for an Extensible Markup Language (XML)
file format used to create common information formats and share both the
format and the data on the World Wide Web, intranets, and elsewhere using
standard ASCII text. XML describes the content in terms of what data is
being described. For example, the word "phonenum" placed within markup
tags could indicate that the data that followed was a phone number. An XML
9

file can be processed purely as data by a program or it can be stored with

similar data on another computer or it can be displayed, like an HTML file.
For example, depending on how the application in the receiving computer
wanted to handle the phone number, it could be stored, displayed, or dialed.
The full format specifications can be read in the IEEE publication from 2007,
at ARFTG Conference, 69th (link to the file)
Inside the broker application, the data from the XML format is converted
to JSON format and stored inside a JSON file for later utilization.
JSON (JavaScript Object Notation) is a lightweight data-interchange
format. It is easy for humans to read and write. It is easy for machines to
parse and generate. It is based on a subset of the JavaScript Programming
Language, Standard ECMA-262 3rd Edition - December 1999. JSON is a
text format that is completely language independent but uses conventions
that are familiar to programmers of the C-family of languages, including C,
C++, C#, Java, JavaScript, Perl, Python, and many others. These properties make JSON an ideal data-interchange language. [5]
JSON is built on two structures:
A collection of name/value pairs. In various languages, this is realized as an object, record, struct, dictionary, hash table, keyed list, or
associative array.
An ordered list of values. In most languages, this is realized as an
array, vector, list, or sequence.
Another important functionality is to perform all the IO operation asynchronously. These means that reading from file (for example) must be performed in the same time with the writing into the file or in the same time
with receiving a message one the network side. Thus, one IO operation must
not block the flow of the rest of the logic of the program. The reason why this
must be done is because of the speedup of the performance. IO operations
are usually the slowest operations in a computational system. The implementation of the asynchronism is performed using Java Threads altogether
with custom anonymous classes implementing Callable Java Interface.
In the case of Threads, they will run continuosly until the end of the
process. It was created a thread for each of the application entities: listener(network), sender(network), writer(file), reader(file). This is valid for
App class. Regarding Broker class, there were used threads for running two
types of entities: Producer and Consumer.
As we have a shared resourse (in our case these are messages which must
be received and sent later on), there appears the Producer-Consmer problem
10

while implementing it asynchronously. In order to solutionate this problem,

there must be used a lock on the requested resource, or to use the thread-safe
datatypes which the programming language offers.
Regarding the current implementation, for communication between threads
was used the BlockingQueue Java class. This is a thread-safe class, whose
methods are atomic (cannot be intrerupted until the operation is completed
and resourse unlocked). Moreover, in order to avoid the exceptions of the
blocked resourse, this class has two specific methods for putting and retrieving from queue - they will wait until the resourse becomes again available.
In order to implement the XML to JSON convertion, it was used a thirtd
party library which can be integrating with Java 1.8. JSON-java library
https://github.com/douglascrockford/JSON-java.
One more utility of the system is the subscribtion of the App entities
for communication with Broker entity. Each entity, before sending a message, must send a subscribtion message to the public IP and port of the
Broker entity. The Broker cannot send a message to an App which was not
subscribed.
In the case that the Broker is not responding and doesnt process the
message, it must be provided a backup. For this purpose was implementet
the clone of the Broker which is ready to take its place. The new entity
is named Replica. Replica sends periodically ping messages to Master.
If the response is not received after a specific amount of time, this means
that Master is down and Replica is changing its state to Master and start
listening to the messages comming from App entities. The problem which
occures is that the new Master cannot resend the messages until the App
entities dont subscribe again. Moreover, the new Master cannot send a
subscribtion request because it doesnt know the addresses of the App.
For future implementation: App entity must resend subscribe message
periodically or in the case that it doesnt receive a MessageDelivered confirmation from Broker.

2.3
2.3.1

Use cases
Scenarios

Using the system, the user is able to send a read the content of an XML
file, send it to a differents user through the network, receive a confirmation
that the message arrived, receive a message from a different user, store the
content of the message in an XML file.
First scenario - read from file.
11

1. User selects the file which he wants to read.

2. User tries to open the file.
3. User tries to read the content of the file.
4. User closes the file.
Second scenario - send through the network.
1. User set the id of the receiver of the message.
2. User creates the message object.
3. User tries to send the message through the network connection.
4. User receives a confirmation from the broker that the message was
delivered.
Third scenario - write into the file.
1. User save the body of the message he wants to save.
2. System tries to search the directory path specified by user.
3. System writes the content of the message into the file.
Alternate scenario 3, starting with step 2) through step 3).
3. If the path was not found, system creates the necessary directories for
fulfilling the path specification.

12

2.4
2.4.1

Design and application architecture

Class Diagram

Figure 4: Class Diagram.

13

2.4.2

Component Diagram

Figure 5: Component Diagram.

Code

The code for realizing the Producer on the broker side:

1 public class Producer implements Runnable {
2
3
private BlockingQueue < Message > queue ;
4
private int port ;
5
private boolean replica ;
6 //
private BlockingQueue < Subscriber > subscribers = new
LinkedBlockingQueue < Subscriber >() ;
7 //
private ExecutorService executor =
Executors . newCachedThreadPool () ;
8
9
Producer ( BlockingQueue q , int port , boolean replica ){
10
queue = q;
11
this . port = port ;
12
this . replica = replica ;
13
}
14
15
@Override
16
public void run () {
17
// listen to the messages
18
NetworkIO netRead = new NetworkIO () ;

14

19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39 }

netRead . setPort ( this . port );

IAsyncIO fileRead = new FileIO () ;
Message mess ;

try {
while ( true ) {
mess = netRead . read (" localhost ");
if (! mess . getType () . equals (" change_port ")){
queue . put ( mess );
} else {
this . port = 3000;
netRead . setPort ( port );
}
}
} catch ( InterruptedException ex ) {
Logger . getLogger ( Broker . class . getName () ).
log ( Level . SEVERE , null , ex );
}

When the broker is created, it must be sent an optional parameter of

boolean type which will define if the given broker is the main one or replica.
The Producer class must implement the Runnable interface because it will
be put inside a new thread inside broker. A broker has mainly one Producer
and some Consumers. The purpose of the Producer is to listen for new
messages and to put them inside the shared BlockingQueue. The Consumer
will retrieve messages from the queue and will process them later on.
Regarding the Consumer, one of the important steps is the registration
of the subscribtions. And here it is performed:
1
2
3
4
5
6
7
8
9
10
11
12
13
14

void subscribeApp ( SubscribtionParameter params ) {

String appName = params . getAppID () ;
String ip = params . getIp () ;
int port = params . getPort () ;

Subscriber rcvr = new Subscriber ( appName , port , ip );

try {
this . subscribers . put ( rcvr );
} catch ( InterruptedException ex ) {
Logger . getLogger ( Consumer . class . getName () ).
log ( Level . SEVERE , null , ex );
}

15

For storing the subscribers it was used again a BlockingQueue<Subscriber>

- subscribers. This was done because of the concurent programming and
the problem of the shared resource between the Consumer threads. For cummunication between the Producer and Consumer threads, it was used another
BlockingQueue<Message> - queMessage.
The main utility of the Broker is to resend the message, and so it does
using the following method:
1 synchronized void sendMessage ( Message mess , NetworkIO netWriter ) {
2
String to = (( MessageParameter )
mess . getParams () ). getReceiverID () ;
3
Iterator < Subscriber > it = this . subscribers . iterator () ;
4
5
if ( this . subscribers . isEmpty () ) {
6
System . out . println (" The subs q is empty ");
7
}
8
9
while ( it . hasNext () ) {
10
Subscriber sub = it . next () ;
11
System . out . println ( sub . name );
12
if ( to . equals ( sub . name )) {
13
netWriter . setPort ( sub . port );
14
netWriter . write (to , mess );
15
System . out . println (" Resending the message to " +
to );
16
return ;
17
}
18
}
19
20
// save the message into a file
21
System . out . println (" The receiver wasn 't found .");
22
}

Another problem which may be encountered is the undelivered messages.

In order to track those messages, the messages are stored inside some files,
apriori transforming the XML content to JSON format. We will store all the
information inside the MessageFile class. And so, there will be created a
collection of these objects. Another thread on the Broker side will periodically check the status of the objects from this collection and will delete the
delivered files, thus remaining only the undelivered ones.
1
2
3
4
5
6

MessageFile messFile = new MessageFile () ;

MessFile . setParams ( messParam );
messFile . setDelivered ( false );
final String filePath = " src / broker /" + messParam . getSenderID ()
+ "_" + messParam . getMessID () + ". json ";

16

7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37

Message copyMessage = message ;

String body = message . getBody () ;
try {
JSONObject xmlJSONObj = XML . toJSONObject ( body );
body = xmlJSONObj . toString (4) ;
// System . out . println ( body );
} catch ( JSONException je ) {
System . out . println ( je . toString () );
}
copyMessage . setBody ( body );
final Message finalMessage = copyMessage ;
Callable < Void > callable ;
callable = new Callable < Void >() {
@Override
public Void call () throws Exception {
fileWrite . write ( filePath , finalMessage );
return null ;
};

Future < Void > futureFile = fileWriter . submit ( callable );

messFile . setFilePath ( filePath );
messFile . setFileWrite ( futureFile );

The rest of the code can be accessed by the repository link on github:
https://github.com/ViSilver/Message-Broker.

17

Conslusion

Performing the given laboratory work I was able to implement a simple

Message Broker. I got acustomed with UDP Protocol and the basics of
the messaging communication systems. Following the task description, I
was able to build a system which consists of different components that may
be changed in future by their new implementations. The system is low
coupled, which means that the interdependecies between the component was
reduced. The system in the end is scalable, which means that the number of
senders and receivers may grow. The system is fault tolerant because there
is a backup Broker in the case that the first one fails. This feature has a
big advantage because in this manner we can grow the performance of our
application. Another feature of scalability is by increasing the number of
consumers. Thus, if the messages will become hard to process, or it will
take too much time, the number of consumers can be grown to increase the
performance. This application may be used for sending data between two or
more entities.

18

References
[1] Gregor Hohpe, Bobby Woolf, Enterprise Integration Patterns. Designing, Building and Deploying Messaging Sollutions, 2012.
[2] Andrew S. Tanenbaum, Prentice Hall, Computer Networks, 1999.
[3] Ren Spronk - Sr.Consultant, Ringholm bv, The role of Message Brokers: a management overview, October 2003.
[4] George Coulouris, Jean Dollimore, im Kindberg, Gordon Blair, DISTRIBUTED SYSTEMS. Concepts and Design, Fifth Edition, 2012.
[5] http://www.json.org/

19