Chapter 2

Chapter 2
Communication
Hsung-Pin Chang
Department of Computer Science
National Chung-Hsing University
PDF created with FinePrint pdfFactory Pro trial version www.pdffactory.com

Outline
o Layered Protocols
o Remote Procedure Call (RPC)
o Remove Object Invocation
o Message-Oriented Communication
o Steam-Oriented Communication

Chapter 2.1 Layered Protocols
o Introduction
o Lower-layer Protocols
n Physical layer
n Data link layer
n Network layer
n Transport layer
o Higher-level Protocols
n Session and Presentation Protocols
n Application Protocols
n Middleware Protocols

Introduction
o Interprocess communication is the heart of all
distributed systems
n Modern DS consists of x1000 or more processes scattered
across an unreliable network
o All communications in DS is based on exchanging
low-level messages
o However, we need to agree on some assumptions
n Volts for 1 or 0?
n First and last bit of a message?
n Damaged or lost message?

Introduction (Cont.)
o Solutions: protocol
n A protocol defines the format and the order of messages sent and
received among network entities, and the actions taken on message
transmission and receipt
o Human protocols:
n “What’s the time?”
n “I have a question”
o Network protocols:
n machines rather than humans
n all communication activity in Internet governed by protocols
o A collection of protocols is called a protocol stack or
protocol suite

Introduction (Cont.)
o Standard : OSI model (or ISO OSI)
n ISO: International Standards Organization
n OSI: Open Systems Interconnection Reference
Model
n Allows open systems to communicate
n Consist of sever-layers

OSI 7-layered Model

Message Encapsulation

Lower-Layer Protocols
o Physical Layer
o Data Link Layer
o Network Layer
o Transport Layer

Physical Layer
o Contain the specification and implementation
of bits and their transmission between sender
and receiver
n Concerned with transmitting 0s and 1s
n How many volts to use for 0 and 1
n Uni- or bi-directional transmission
n Transmission speed (bits/sec)

Data Link Layer
o Prescribe the transmission of a series of bits into a
frame (packet) to allow for error and flow control.
n Grouping bits into frames and make sure each frame is
correctly received
n By putting a special bit pattern on the start (header) and
end of each frame (checksum)
n Computing checksum
o Examples
n Ethernet, IEEE 802.11 WLAN

Network Layer
o Describes how packets in a network of
computers are to be routed
n It may take several hops from a sender to a
receiver
n Routing is the primary task of this layer
n The most popular protocol: IP (Internet Protocol)
n ATM virtual channel is another example

Transport Protocols
o The last part of a basic network protocol stack
o Provides the actual communication facilities
for most distributed systems
o Standard Internet protocols:
n TCP: connection-oriented, reliable, stream-
oriented communication
o But high-overhead
n UDP: unreliable (best-effort) datagram
communication
o But low-overhead

Client-Server TCP
o In client-server architecture
n The clients may only send a request
n Then the server respond with a answer
o However, TCP must manage the connection
n Cause too much overhead
o Solution: TCP for Transactions (T/TCP)
n Combine the connection management with the
request/answer into one packet

Client-Server TCP
Normal TCP Transactional TCP

Higher-Level Protocols
o Session layer: an enhanced version of transport layer
n Provide dialog control
n Keep track of which party is currently talking
n Provide synchronization facilities
o Presentation layer: different machine have different
data representation, thus we need
n Conversion, compression, or encryption
o Application layer
n FTP
n HTTP……

Higher-Level Protocols (Cont.)
o However, only the application layer is ever
used
n Everything above the transport layer is grouped
together
n The session layer, presentation layer are included

in the application layer of Internet protocol suite
(TCP/IP).

Higher-Level Protocols (Cont.)
o Middleware protocols: protocols to support a variety
of middleware services
n Slide. 61 in Chapter 1
o Examples:
n Authentication protocols
n Distributed commit protocols
n Distributed locking protocols
o Middleware protocols support high-level
communication services, like RPC, Remote Object
Invocation, message queuing services…

Middleware Protocols

Chapter 2.2: Remote Procedure Call
o Basic RPC operations
o Parameter Passing
o Extended RPC Models
o DCE RPC

Introduction
o RPC (Remote Procedure Call)
n Allows programs to call procedures located on
other machines
n Not by calling primitive send() and receive()
o Problems
n Different address spaces à pointer?
n Data representations à integer, floating?
n Machine crashes?

Basic RPC Operations: Fig. 2.6
o Conventional Procedure Call
n count = read(fd, buf, nbytes)
the stack before the call to read The stack while the called procedure is active

Basic RPC Operations (Cont.)
o Problem
n Call-by-value or call-by-reference
o fd, and nbytes are call-by-value
n Their value is copied into the stack
n Modification in the called procedure but no changes in the
calling procedure
o buf is call-by-reference
n Its address is copied into the stack
n Modification in the called procedure is reflected in the
calling procedure

Client and Server Stubs
o The main challenge of RPC is to look as
much like LPC, i.e., transparency
o Transparency is achieved via “stubs”

n A library in the client (client stub) and server
(server stub)
n To be linked with the program

Client and Server Stubs (Cont.)
o Client stub: is called using the conventional calling sequence
(Fig. 2.6 in previous two slide)
n Packs parameters into a message
n Calls OS to send directly to the server
n Waits for result-return from the server by calling receive()
o Server stub
n Be blocked waiting for incoming message by calling receive()
n Unpacks the parameters
n Calls the corresponding procedure in the usual way (Fig. 2.6)
n Returns results to the caller
n Does a call to receive again to wait for next request

Client and Server Stubs

RPC Steps
1. Client procedure calls client stub in normal way
2. Client stub builds message, calls local OS
3. Client's OS sends message to remote OS
4. Remote OS gives message to server stub
5. Server stub unpacks parameters, calls server
6. Server does work, returns result to the stub
7. Server stub packs it in message, calls local OS
8. Server's OS sends message to client's OS
9. Client's OS gives message to client stub
10. Stub unpacks result, returns to client

Parameter Passing
o Packing parameters into a message is called parameter
marshaling
o It’s fine if
n Client and server machine are identical
n And all the parameters and results are scalar types, like
integers, characters, and booleans.
o However, possible problems:
n IBM mainframes use EBCDIC char code and IBM PC uses
ASCII code
n Integer: one’s complement and 2’s complement
n Floating-point numbers
n Little endian and big endian

Different Data Representations
o (a) Original message on the Pentium (little endian) (5, “JILL”)

n Number their bytes from right to left
o (b) After receipt on SPARC (big endian) (5x2^24, “JILL”)
n Intel sends the message by: 0, 1, 2……., But SPARC puts in big
endian order
o (c) Solution but wrong: invert the bytes
n The message after being inverted. (5, “LLIJ”)
o The little numbers in boxes indicate the address of each byte
Passing Value Parameters
o Execution of add(i,j) via RPC

Passing Reference Parameters
o A pointer is only meaningful in its address space
o Possible solutions
n Forbid pointer usage in RPC calls
n Copy the entire pointed area (such as arrays or strings) into
the message and send it to the server
o i.e., call-by-reference is replaced by call-by-copy/restore
n Not always identical but is good in most cases
o Only suitable for bounded and known areas
o Optimization: distinguish input parameter and output parameter
n Input parameter: like write, it need not be copied back
n Output parameter: like read, need not be sent over

Parameter Specification and Stub
Generation
o Parameter specification
n Agree on the format of the message they are exchange
o Word: 3 bytes……
n Agree on the representation of data structure
o Integer: two’s complement
n Agree on the actual exchange of message
o TCP or UDP
o Parameter Marshalling
n Wrapping parameters into a message with proper coding

Parameter Specification and Stub
Generation (Cont.)
o Stub Generation
n Stub for different procedure differ only in their
interface to the applications
n Interface
o A collection of procedures
o Often specified by an Interface Definition Language
(IDL)
n Interface specified in IDL is then compiled into a
client stub and a server stub

Note: call-by-copy/restore
o Not common in modern programming language
o The parameters is copied to the stack and then
copied back after the call
n Overwrite the caller’s original value
o Achieve the same effect as call-by-reference under
most conditions
o But in some cases, like the same parameter being
present multiple times in the parameter list
n The semantics are different

Extended RPC Models
o Doors
n RPC for processes located on the same machine
n In some OS, it is called lightweight RPC
o Asynchronous RPC
n Clients immediately continues after issuing a RPC request, i.e.,
without blocking
o Deferred synchronous RPC
n A client and server interacting through two asynchronous RPCs
n Server will call the client when the processing is done
o One-way RPCs
n Client continue immediately after calling the server
o Do not wait for an acknowledgement from the server
n Reliability ?

Doors
OS upcalls the procedure
(4)
(1)
(2) Associate the fd with a name

(3)
(5)

Asynchronous RPC
(a) The interconnection between client and server in a traditional RPC

(b) The interaction using asynchronous RPC

Deferred Synchronous RPC
A client and server interacting through two asynchronous RPCs

Example: DCE RPC
o DCE: Distributed Computing Environment
developed by OSF (now called Open Group)
n A true middleware system
o Support for UNIX, VMS, Windows NT
n Idea:
o Add DCE on top of existing machines to behave as an distributed
system (p. 61 in Chap. 1)
n Programming model: client-server model
o All communication between client and server is by RPC
n Since RPC system hide the details, client and server are independent
o Client in Java and Server in C
o Client and server can run on different hardware or different OS

Example: DCE RPC (Cont.)
n A number of services included
o Distributed file service
o Directory service
n Keep track of the location of all resources
o Security service
o Distributed time service

n Keep clocks on different machines globally synchronized

Coding Sequence in DCE
A program used to generate an identifier and prototype IDL file

DCE RPC Coding Sequence
o Hold everything together: IDL (Interface Definition
Language)
n IDL files contain function prototype, type definition, constant
declarations, and info needed for marshalling and unmarshall
o Step1: use the uuidgen program
n A program used to generate a prototype IDL file containing
a globally unique identifier
o The client sends this ID in its first RPC to the server to prevent
inadvertently binding
n If sent to the wrong server, the server can detect the error
o The ID is obtained by uuidgen program
n Cannot be used again in any other interface file
n Guarantee by encoding ID from location and time of creation

DCE RPC Coding Sequence (Cont.)
o Step2: edit the IDL file to fill in names and parameters of RPCs
o Step3: compilation and the output has three files
n A header file
o The unique identifier, type definition, constant definition, and function
prototype
o Included in both client and server code
n The client stub
o Contain the actual procedure that the client program will call
n The server stub
o Contain the procedure called by the runtime system on the server machine
o Step 4: write the client and server code and compiler them
o Step 5: link the client code, client stub with runtime library. So
as server

Binding Client to Server in DCE RPC
o How does the client locate the server?
o Binding
n A process of determining the remote procedure
and the machine on which it executes

Binding Client to Server in DCE RPC
Provide (a) network address of server

by server’s name (b) server’s name
3’. return the server IP address

endpoint=port
reply endpoint
A table of (server, endpoint)

Performing an RPC
o DCE provides several semantic option
o At-most-once operation
n No call is ever carried out more than once
n RPC operation is guaranteed to have been carried out at
most once, but possibly none at all.
o Like server crashes, etc
o Idempotent
n Declared in IDE file
n It can be repeated multiple times without harm
o Repeat when previous call fails
o Like reading a specified record

Chapter 2.3: Remote Object Invocation
o Distributed Objects
o Binding a Client to an Object
o Static/Dynamic RMI
o Parameter Passing
o DCE Remote Objects
o Java RMI

Objects
o Object hides its internals from outside world
through a well-defined interface
n Allow object to be easily modified, as long as the
interface remains the same
o Object includes
n State: consists of values of its instance variables,
o i.e., encapsulate the data
n Methods: operations on those data.
o Methods are made available through interface

Distributed Objects
o Distributed objects
n Place an interface at one machine, while the
object itself resides on another machine
o An object’s interface includes
n The method of an object for invocation by other
objects in other processes
n The type of input and output parameters
o “Object” can also be passed as parameter
o Object reference would also be used as parameter

Distributed Objects
o Notably, Java and C++ allow an instance
variable to be accessed directly from outside
world
n Not permitted in a distributed object system
o An object’s data should be accessed only by its
method

Distributed Objects
Organization of a remote object with client-side proxy

Distributed Objects (Cont.)
o When a client binds to a distributed object
n A proxy must be in the client’s address space
n Proxy: client-side stub
n Skeleton: server-side stub
o Remote object: one kind of distributed objects
n State of an object is not distributed
o It resides at a single machine
n Only interfaces are made available

Compile- time versus Runtime Objects
o Compiler-time object:
n Java, C++, or object-oriented languages
n An object is defined an instance of a class
n Easier to build distributed application
n But dependent on a particular programming lang.
o Runtime objects
n Construct distributed objects during runtime
n Independent of programming language
o App. can be built from objects written in different PL
n But how object are implemented?
o Object adapter: a wrapper around the implementation to give it (like C
function) the appearance of an object
o Allow an interface to be converted into something that a client expects

Persistent and Transient Objects
o Persistent object
n An object that is guaranteed to live between activations of
server processes
o Persistent object is not dependent on its current server process
n Sol. its state can be stored in secondary storage
o A server process exits and store the state in the disk
o After a while, a new server process start and read in the state
o Transient Objects
n Live as long as the invoking server does

Implicit & Explicit Binding
o Implicit binding
n Allows a client to directly invoke methods using
only a reference to an object
o Explicit binding
n The client should call a special function to bind to
the object before it actually invoke its methods

Implicit & Explicit Binding
Distr_object* obj_ref; //Declare a systemwide object reference
obj_ref = …; // Initialize the reference to a distributed object
obj_ref-> do_something(); // Implicitly bind and invoke a method
(a)
Distr_object objPref; //Declare a systemwide object reference

Local_object* obj_ptr; //Declare a pointer to local objects
obj_ref = …; //Initialize the reference to a distributed object
obj_ptr = bind(obj_ref); //Explicitly bind and obtain a pointer to the local proxy
obj_ptr -> do_something(); //Invoke a method on the local proxy
(b)

Implementation of Object Reference
o Object reference
n An identifier that can be used as an address of a remote
object
o An object reference should have information to allow
a client to bind to an object
n IP address (machine), endpoint (port), and object identifier
(object)
Representation of an Object Reference
32 bits 32 bits 32 bits 32 bits
Internet address port number time object number interface of

remote object

(Cont.)
o Problems
n Server crashes à assign a new endpoint à all references
become invalid
o Sol: local daemon per machine listen to a well-known endpoint
n Keep track of the server-to-endpoint assignments in a table
n However, if server is moved to another machine, the local
daemon approach is not valid
o Sol: location server maintaining where object’s server is running
o Problem: scalability issue

(Cont.)
o Problem (Cont.)
n Protocol assumption à client and server must use
the same protocol stack
o Sol: include implementation handle in the object
reference
n e.g. ftp://ftp.clientware.org/proxies/java/proxy-v1.1a.zip
n From implementation handle, client can dynamically
downloaded the needed proxy
n Client need not worry about whether it has an
implementation of a specific protocol available

Static and Dynamic Remote Method
Invocations
o RMI (Remote Method Invocation)
n Invoke an object’s method through proxy
o Static invocation
n Use predefined interface definitions
o Either use interface definition language like RPC approach
o Or use object-based language (Java) that generate stub
automatically.
n A change to the interface requires all applications (i.e.
clients) to be recompiled
n objectname.methodname(para)

Static and Dynamic Remote Method
Invocations (Cont.)
o Dynamic invocation
n Dynamic invocation “composes” a method invocation at run-time
n An application selects at runtime which method it will invoke at a
remote object
n invoke(object, method, inpars, outpars)
n Usage example: implement a object browser to examine sets of object
o Dynamically present object’s interface to the user
o User then select which method to invoke and show the parameters
o Example: an object called fobject with method append
n Static: fobject.append (int)
n Dynamic: invoke(fobject, id(append), int)

Parameter Passing
o When invoking a method with an object reference as
parameters
n For efficiency, differentiate such an object reference is
referred to a local object or a remote object
o For remote object: must issue request between machines,
inefficient when objects are small (integers or Booleans)
o If object reference refers to a local object: passed by
value
n The referred object is copied to the server as parameter
o If object reference refers to a remote object: passed by
reference
n The reference is copied and passed as a parameter

Parameter Passing
Client program at A invoke method of object running at C.
Two object references to O1 and O2 are passed along as parameters
But reference to O1 is a local object while to O2 is remote
So that object at
C can access O1
directly=>
efficient
The situation when passing an object by reference or by value.

Note: O1 is passed by value; O2 is passed by reference.
Example 1: DCE Remote Objects
o DCE’s RPC mechanism has been “enhanced” to
directly support remote method invocation
n An extension to their RPC model that is refined from RPC
o DCE Objects = xIDL plus C++.
n Distributed object are specified in IDL and implemented
in C++
o DEC distributed objects take the form of remote
objects
n A server is responsible for creating C++ objects locally
and making methods available to remote client
n There is no way to create distributed objects

Note: Remote Objects
o A distributed object whose state always
resides on a single machine
n But whose interface can be made available to
remote process

(Cont.)
o Two types of DCE objects are supported
n Distributed Dynamic Objects – a “private” object
created by the server for the client.
o Server creates objects locally on behalf of a client and
is only accessible to the client
n Distributed Named Objects – a “shared” object
that lives on the server and can be accessed by
more than one client
o Created by server and shared by clients

The DCE Distributed-Object Model
a) DCE dynamic objects – requests for creation sent via RPC.

b) DCE named objects – registered with a DS naming service.
(Cont.)
o Problem in DCE objects
n No mechanism for transparent object reference
o Since refine from RPC, each object invocation is done by RPC
o Thus, lacking a proper system-wide object reference mechanism
n Transparency: client only identifies a remote procedure
in a server
o But in DCE, client identifies a remote procedure in a server’s
object
n Pass the object identifier, the interface identifier, and parameters

Java Distributed-Object Model
o Java also uses remote objects as the only form of
distributed objects
o Java intends to be transparent => making remote object
access similar to local object
o But network, server machine or server process may fail
o Thus, difficult since difference exists between remote
object and local one
n Cloning local or remote object is different
n Semantics of blocking on an local or remote object

Java Distributed-Object Model (Cont.)
o Cloning objects:
n Cloning local object result in a new object of the same
type with the same state at the local machine
n However, semantics of remote object cloning is different
o If client want to clone a remote object
n Clone the actually object at its server + clone all proxies that are
currently bound to the remote object at each client (bad)
o Thus, remote object can be cloned only by server (to server-> its
local object)
n Proxies are thus not cloned
n That is, a remote object is not allowed to be cloned by client

Java Distributed-Object Model (Cont.)
o Semantics of blocking on an local or remote object
n Java supports objects to be as monitor
o By declaring a method to be synchronized
o Only one process can proceed if two processes call a
synchronized method
o The other process is blocked inside a (local) object waiting for a
conditional variables
n But how to blocking on remote object if invoking a
remote synchronized method
o Sol 1: allowing blocking only at server
n What happened when a client crashes when its invocation is being
handled by the server?
o Sol 2: blocking at proxy, i.e., client-side stub
n Need to synchronize different clients at different machine (complex)

Java Remote Object Invocation
o Any primitive or object type can be passed as
a para to an RMI only if it can be marshaled
n Which is called serializable in Java terminology
n Platform dependent object, like file descriptors
and sockets, cannot be serialized
o Like DCE, local objects are passed by value
and remote objects are passed by reference

Java Remote Object Invocation (Cont.)
o Proxy itself is serializable (proxy just likes local obj.)
n Can be marshaled, sent, then unmarshaled remotely and
used to invoke method on the another remote object
o A proxy can be used as a reference to a remote object
o i.e., Allow remote invocation at any machine
n Distinguishing features of Java RMI
n This is possible only when each process is executed the same Java
virtual machine, i.e., the same execution environment
o DCE does not allow stub passing
o DCE allows difference processes at difference languages, operating
system and hardware platform

Java Remote Object Invocation (Cont.)
o A remote object is built from two classes
n Server class: implementation of server-side code
o Contain the object’s state and method
o Skeleton: server-side stub, is generated from the interface
specifications of the object
n Client class: implementation of a proxy
o Also generated from the object’s interface spec.
o Convert each method call into a message and a reply message
into the result of a method call
n For each call, proxy also sets up a connection with a server
o Thus, it has all the information to let client finds remote object
n Server’s network address, endpoint, identifier of the object

Chapter 2.4 Message-Oriented
Communication
o Persistence and Synchronicity
o Message-Oriented Transient Communication
o Message-Oriented Persistent Communication
o IBM MQSeries

Introduction
o As a communications mechanism, RPC/RMI is often
inappropriate.
n The receiving side must be executing at the time a request
is made.
n The inherent synchronous mode of their operation forces
the client to be blocked
o However,
n What happens if the receiving side is not “awake”?
n In addition, the default “synchronous, blocking” nature of
RPC/RMI is often too restrictive.
o Something else is needed: Messaging

Persistence and Synchronicity
o Persistent Communications
n Once sent, the “sender” can stop executing
n The “receiver” need not be operational at this time
n The communications system buffers the message as
required (until it can be delivered).
n E.g. email system:
o Transient Communications
n Message is stored in comm system only when both sender
and receivers are executing
n If problems occur, the message is simply discarded
n E.g. routing

Message-Oriented Communication

Persistence and Synchronicity (Cont.)
o Synchronous Communications
n A sender blocks, waiting for a reply from the
receiver before doing any other work
o Asynchronous Communications
n A sender continues with other work immediately
upon sending a message to the receiver

Persistence and Synchronicity: Six
Cases
o Persistence and Synchronicity lead to six
combinations
o (a) Persistent asynchronous communication
n Msgs are stored at the local host or at the communication
server, ex., email
o (b) Persistent synchronous communication
n Msgs are presistently stored only at the receiving host
n Sender is blocked until its message is stored in receiver’s
buffer
o Not necessary the receiving application is executing to store the
message

Classifying Distributed Communications (1)
o Persistent asynchronous communication.

o Persistent synchronous communication.

Cases (Cont.)
o (c) Transient asynchronous communication
n Typically offered by transport-layer datagram, like UDP
n Msgs are stored at the local buffer, the sender continues,
and route by the comm. system.
n But the comm. fails if the receiver is not currently running
o (d) Receipt-based transient synchronous
communication
n Weakest form of synchronous comm
n The sender is blocked until the msg is stored at the recvr’
local buffer at the receiving host
n The sender recvs an ACK and continues

c) Transient asynchronous communication.

d) Receipt-based transient synchronous communication.
Cases (Cont.)
o (e) Delivery-based transient synchronous
communication
n The sender is blocked until the message is delivered to the
receiver for further processing
n Asynchronous RPC follows this policy
o (f) Response-based transient synchronous
communication
n The strongest form of synchronous behavior
n The sender is blocked until it receives a reply from the
other side
n RPC and RMI adhere to this

e) Delivery-based transient synchronous communication at message

delivery.
f) Response-based transient synchronous communication.
Message-Oriented Transient
Communications
o Transient Messaging Examples
n Sockets
n Message-Passing Interface (MPI)

o Used for Clusters of Worksations, Massively Parallel
Processors, often for scientific applications.
n Remote Procedure Call/ Remote Method

Invocation

Berkeley Sockets
Primitive Meaning
Socket Create a new communication endpoint
Bind Attach a local address and port number) to a socket

Announce willingness to accept connections. OS will
Listen
reserve buffer
Accept Block caller until a connection request arrives
Connect Actively attempt to establish a connection
Send Send some data over the connection
Receive Receive some data over the connection
Close Release the connection

Berkeley socket API

Socket Operations
Connection-oriented communication pattern using sockets.

Message-Passing Interface (MPI)
o However, DS developers rejected Sockets:
n Wrong level of abstraction with only “send” and “receive”.
n Closely coupled to TCP/IP networks
o Not suitable for proprietary protocol for high-speed interconnection
network.
o Middleware vendors thus provide a higher-level of
abstraction
n Every vendor did their own and cause portability problems
o Solution: The “Message-Passing Interface” (MPI).

Message-Passing Interface (MPI)
(Cont.)
o MPI is designed for parallel applications and
thus tailed to transient communication
o MPI assumes that comm takes places within a

known group of processes
n (groupID, processID) instead of a transport-level
address in socket

Some MPI Primitives
Primitive Meaning
MPI_bsend Append outgoing message to a local send buffer
MPI_send Send a message and wait until copied to local or remote buffer
MPI_ssend Send a message and wait until receipt starts

MPI_sendrecv Send a message and wait for reply
MPI_isend Pass reference to outgoing message, and continue
MPI_issend Pass reference to outgoing message, and wait until receipt starts
MPI_recv Receive a message; block if there are none

MPI_irecv Check if there is an incoming message, but do not block
Some of the more intuitive (and useful) message-passing primitives

(Note: there are many more in the API).

MPI Primitives
o MPI_bsend =
n Transient asynchronous comm
o MPI_send, the semantics depends on implmentation
n MPI_senduntil copies to local buffer =
o Receipt-based transient synch comm
n MPI_send until copies to remote buffer =
o Delivery-based transient synch comm
o MPI_ssend =
n Delivery-based transient synch comm
o MPI_sendrecv =
n Response-based transient synch comm

Message-Oriented Persistent
Communication
o Also known as: Message Queuing Systems,
or Message Oriented Middleware (MOM)
o Support persistent, asynchronous comms
o Basic idea: processes communicate through
support of middleware queues.
n Queues are buffers at communication servers.
o Key Example: IBM MQSeries

Message-Queuing Models
o It offers intermediate-term storage capacity for messages
n Without requiring either the sender or receiver active during message
transmission
n Apps communicate by inserting messages into specific queues
n Forwarded to destination, even if dest. down when message was sent.
o Addressing by a system-wide unique name of the destination
queue
o Only guarantee: your message will eventually make it into the
receiver’s message queue
n No guarantee about when, or even if the message will be actually read
o Typically, transport can take minutes as opposed to
seconds/milliseconds in Sockets or MPI

Message Queuing Interface
Primitive Meaning
Put Append a message to a specified queue (nonblocking)
Block until the specified queue is nonempty, and remove
Get
the first message (blocking)
Check a specified queue for messages, and remove the
Poll
first. Never block. (nonblocking)
Install a handler in receiver to be called when a message
Notify
is put into the specified queue. (callback function)
Basic interface to a queue in a message-queuing system: this is a very

simple, yet extremely powerful abstraction.

Message-Queuing Models System is storing (and possibly transmitting)
message while sender and receiver are passive
Four combinations for “loosely-coupled” communications which use

message-queues.

General Architecture of a Message-
Queuing System
o Messages are put into a source queue
o MOM must route to a destination queue.
o Queues must be mapped to network locations.
n A database of queue names to network locations, see
next Figure
n Problem
o No general naming service in many message-queuing system to
maintain queue-to-location mapping
o Each queue manager needs a copy of the queue-to-location
mapping
n Lead to management problem
o Queue Manager: manage the queue

General Architecture of a Message-
Queuing System (Cont.)
o Sol. Relay: special queue manager operates as
router
n Instead of using a naming service that maintains a
queue name to network location mapping
n See next two Figures

General Architecture of a Message-Queuing
System (1)
The relationship between queue-level addressing and network-level

addressing.

General Architecture of a Message-Queuing
System (2)
The general organization of a message-queuing system with routers

(relays).

Message Brokers
o Often, there’s a need to integrate new/existing apps
into a “single, coherent Distributed Information
System”.
o Problem: different message formats exist in legacy
systems
o It may not be convenient to “force” legacy systems to
adhere to a single, global message format (cost!?).
o It is often necessary to live with diversity.

Message Broker Organization
The general organization of a message broker in a message-queuing

system – also known variously as an “interface engine”.

Message Brokers (Cont.)
o But the message format must be understood by
both sender and receiver
o By the “Message Broker”.
o Message broker: act as an application gateway
to convert incoming messages to a format
understood by the receiver
n By a database of rules

A Note on Message-Queuing System
o Message-queuing is different from email system
n Email is targeted for end users
n MQ is a general-purpose interprocess comm and holds lots
features
o Message priorities, logging facilities, efficient multicasting, fault-
tolerance
o Most important MQ application area:
n The integration of a widely dispersed collection of database
applications (which is impossible to do with traditional
RPC/RMI techniques).

Example: IBM MQSeries
o Popularly used in IBM mainframes
n Especially for finance applications
o All queues are managed by Queue Manager (QM)
o Message Channel: a unidirectional, reliable
connection between a sending and a receiving queue
manager
o Each of the 2 ends of a MC is managed by message
channel agent (MCA)

IBM MQSeries Organization
General organization of IBM's MQSeries message-queuing system.

IBM MQSeries: Message Channels
Attribute Description
Transport type Determines the transport protocol to be used.
FIFO delivery Indicates that messages are to be delivered in the order they are sent.
Message length Maximum length of a single message.

Setup retry
Specifies maximum number of retries to start up the remote MCA.
count
Delivery retries Maximum times MCA will try to put received message into queue.
Some attributes associated with message channel agents

(MCA).

Message Transfer in MQSeries
o An address consists of (name of QM, name of Destination
Queue)
n A QM may consists of many queues
o Routing table entry in QM = (destQM, next_sendQ)
o QM’s name is systemwide unique
n However, when replace a QM or change a QM’s name affects all apps
that send messages to this QM
n Solution: local alias: a logical name of a QU
o By local alias
n Change the name of a QM requires change its alias in all queue
managers
n But the applications can be left unaffected

Alias and Routing Table
Local alias of a QM
QM’s real name

IBM MQSeries: Message Transfer
Primitive Description
MQopen Open a (possibly remote) queue.
MQclose Close a queue.
MQput Put a message into an opened queue.
MQget Get a message from a (local) queue.
Primitives available in an IBM MQSeries MQI.

Chapter 2.5: Stream-Oriented
Communications
o Continuous Media Support
o Quality of Service (QoS)
o Stream Synchronization

Stream-Oriented Communications
o With RPC, RMI and MOM, timing has no effect on
correctness.
o However, audio and video are time-dependent data streams
n If the timing is off, the resulting “output” from the system will be
incorrect.
n Time-dependent information – known as “continuous media”
communications.
o Discrete (representation) media
n E.g. text, still images, object code
o Continuous (representation) media
n Temporal relationships between data are crucial

Transmission Modes
o Data streams: a sequence of data units
o Asynchronous transmission mode
n The data stream is transmitted in order, but there’s no timing
constraints placed on the actual delivery (e.g., file transfer).
o Synchronous transmission mode
n The maximum end-to-end delay is defined (but data can travel faster).
o Isochronous transmission mode
n Data transferred “on time”
n i.e., there’s a maximum and minimum end-to-end delay, known as
“bounded jitter” (e.g., multimedia systems)
n The type of steam discussed in this chapter

Two Types of Streams
o Simple Streams: one single sequence of data
n E.g. voice
o Complex Streams: several sequences of data
(substreams) that are “related” by time
n E.g. a lip-synchronized movie, with sound and
pictures, together with sub-titles
n This leads to data synchronization problems,
which are not at all easy to deal with

Virtual Connections
A stream can be considered as a virtual connection between a source and a sink
Source and sink

are processes
Source and sink

are devices
(a) Setting up a stream between two processes across a network

(b) Setting up a stream directly between two devices
Multicasting Stream
Multiparty communication: a stream is multicast to several receivers

However, each receiver have different QoS requirements.
Thus, we need a filters that adjust the quality of incoming stream

Quality of Service (QoS)
o Time-dependent requirements can be expressed as
QoS requirements
o Definition: “ensuring that the temporal relationships
in the stream can be preserved”.
o QoS is all about three things:
n (a) Timeliness, (b) Volume and (c) Reliability
o But, how is QoS actually specified?
o One approach: expressing QoS by flow specification
n Specify bandwidth requirements, transmission rates, delays..
n Ref. the next slide

Specifying QoS with Flow Specs.
Characteristics of the Input Service Required
maximum data unit size (bytes) Loss sensitivity (bytes)

Token bucket rate (bytes/sec) Loss interval (µsec)
Toke bucket size (bytes) Burst loss sensitivity (data units)
Maximum transmission rate Minimum delay noticed (µsec)
(bytes/sec) Maximum delay variation (µsec)
Quality of guarantee
A flow specification – one way of specifying QoS

Token Bucket Algorithm
o In flow spec. the characteristics of the input stream are
formulated by Token bucket algorithm
n Specify how the stream will shape its network traffic
o Token: a fixed number of bytes that an app is allowed
to pass to network
o Tokens are buffered in a bucket with limited capacity
n Tokens are dropped when bucket is full,
o Data are passed to network at a relatively constant rate

An Approach to Implementing QoS:
Token Bucket Algorithm
The principle of a token bucket algorithm – a “classic” technique for

controlling the flow of data (and implementing QoS characteristics).

Service Required in Flow Specs
o In addition to the temporal relations by token bucket
algorithm, a flow spec. also consists of service requirements
o Loss sensitivity and loss interval
n Specify a maximum acceptable loss rate would be
o Burst loss sensitivity
n How many consecutive data units may be lost
o Minimum delay noticed
n How long the network can delay the delivery of a data unit before the
receiver notices the delay
o Maximum delay variation
n The maximum tolerated jitter
o Quality of guarantee
n How serious the service requirements should be taken (hard, firm, or
soft)

Flow Specifications
o Problem
n An application may simple not know its own
requirements
o So, another solution
n Classify a stream and provide defauls for detailed
flow specification
n For example:
o Specify only whether a stream is for audio or video
o Specify the quality is high, medium, or low

Setting up of a Stream
o After flow specification, DS should allocate resource
to set up a stream, including
n Bandwidth, buffers, and processing capacity
n Which are all needed in order to realize QoS guarantees
o However, it is not easy that a model can
n Specify a QoS parameters
n Generically describe resources in any communication
system
n Translate QoS parameters to resource usage

RSVP
o Resource reSerVation Protocol (RSVP)
n A transport-level protocol for enabling resources reservation in
network routers
o First, senders in RSVP
n Set up a path to receivers.
n Then provide a flow specification and send it to each intermediate node
o RSVP is a receiver-initiated protocol
n Receiver, when ready to receive, places a reservation request along
its upstream path to sender
o Reservation can be lower than sender’s specification
n RSVP process reserves resources if enough
o By admission control module
n RSVP process will translate QoS parameters into things that make
sense to the data-link layer

RSVP Protocol
Check whether the receiver has

permission to make the reservation
The basic organization of RSVP for resource reservation in a distributed system.

Stream Synchronization
o Synchronization
n Maintain temporal relations between streams
o Two synchronization forms
n Between a discrete data stream and a continuous
data stream
o E.g. a slide show with audio
n Between continuous data streams
o E.g. playing a movie (lip synchronization)

Stream Synchronization (Cont.)
o Synchronization mechanism
n Where does the synchronization occur?
o On the sending side or receiving side?
n Application transparency?
o Application take care by itself or by middleware

Receive-Side with Application-Aware
The principle of explicit synchronization on the level data units for multiple streams
(sub-streams).

Receiver-Side with Application
Transparency
The principle of synchronization as supported by high-level interfaces

built as a set of “multimedia middleware streaming services”.

Chapter 2

Uploaded by

Document Information

Copyright

Available Formats

Share this document

Share or Embed Document

Sharing Options

Did you find this document useful?

Is this content inappropriate?

Copyright:

Available Formats

Chapter 2

Uploaded by

Copyright:

Available Formats

Chapter 2

PDF created with FinePrint pdfFactory Pro trial version www.pdffactory.com

PDF created with FinePrint pdfFactory Pro trial version www.pdffactory.com

PDF created with FinePrint pdfFactory Pro trial version www.pdffactory.com

PDF created with FinePrint pdfFactory Pro trial version www.pdffactory.com

PDF created with FinePrint pdfFactory Pro trial version www.pdffactory.com

PDF created with FinePrint pdfFactory Pro trial version www.pdffactory.com

PDF created with FinePrint pdfFactory Pro trial version www.pdffactory.com

PDF created with FinePrint pdfFactory Pro trial version www.pdffactory.com

o Data Link Layer

PDF created with FinePrint pdfFactory Pro trial version www.pdffactory.com

PDF created with FinePrint pdfFactory Pro trial version www.pdffactory.com

PDF created with FinePrint pdfFactory Pro trial version www.pdffactory.com

PDF created with FinePrint pdfFactory Pro trial version www.pdffactory.com

PDF created with FinePrint pdfFactory Pro trial version www.pdffactory.com

PDF created with FinePrint pdfFactory Pro trial version www.pdffactory.com

Normal TCP Transactional TCP

PDF created with FinePrint pdfFactory Pro trial version www.pdffactory.com

PDF created with FinePrint pdfFactory Pro trial version www.pdffactory.com

n The session layer, presentation layer are included

PDF created with FinePrint pdfFactory Pro trial version www.pdffactory.com

PDF created with FinePrint pdfFactory Pro trial version www.pdffactory.com

PDF created with FinePrint pdfFactory Pro trial version www.pdffactory.com

o Extended RPC Models

PDF created with FinePrint pdfFactory Pro trial version www.pdffactory.com

PDF created with FinePrint pdfFactory Pro trial version www.pdffactory.com

PDF created with FinePrint pdfFactory Pro trial version www.pdffactory.com

PDF created with FinePrint pdfFactory Pro trial version www.pdffactory.com

o Transparency is achieved via “stubs”

n To be linked with the program

PDF created with FinePrint pdfFactory Pro trial version www.pdffactory.com

PDF created with FinePrint pdfFactory Pro trial version www.pdffactory.com

PDF created with FinePrint pdfFactory Pro trial version www.pdffactory.com

PDF created with FinePrint pdfFactory Pro trial version www.pdffactory.com

PDF created with FinePrint pdfFactory Pro trial version www.pdffactory.com

o (a) Original message on the Pentium (little endian) (5, “JILL”)

PDF created with FinePrint pdfFactory Pro trial version www.pdffactory.com

PDF created with FinePrint pdfFactory Pro trial version www.pdffactory.com

PDF created with FinePrint pdfFactory Pro trial version www.pdffactory.com

PDF created with FinePrint pdfFactory Pro trial version www.pdffactory.com

PDF created with FinePrint pdfFactory Pro trial version www.pdffactory.com

PDF created with FinePrint pdfFactory Pro trial version www.pdffactory.com

(2) Associate the fd with a name

PDF created with FinePrint pdfFactory Pro trial version www.pdffactory.com

(a) The interconnection between client and server in a traditional RPC

PDF created with FinePrint pdfFactory Pro trial version www.pdffactory.com

A client and server interacting through two asynchronous RPCs

PDF created with FinePrint pdfFactory Pro trial version www.pdffactory.com

PDF created with FinePrint pdfFactory Pro trial version www.pdffactory.com

o Distributed time service

PDF created with FinePrint pdfFactory Pro trial version www.pdffactory.com

PDF created with FinePrint pdfFactory Pro trial version www.pdffactory.com

PDF created with FinePrint pdfFactory Pro trial version www.pdffactory.com

PDF created with FinePrint pdfFactory Pro trial version www.pdffactory.com

PDF created with FinePrint pdfFactory Pro trial version www.pdffactory.com

Provide (a) network address of server

3’. return the server IP address

A table of (server, endpoint)

PDF created with FinePrint pdfFactory Pro trial version www.pdffactory.com

PDF created with FinePrint pdfFactory Pro trial version www.pdffactory.com

PDF created with FinePrint pdfFactory Pro trial version www.pdffactory.com

PDF created with FinePrint pdfFactory Pro trial version www.pdffactory.com

PDF created with FinePrint pdfFactory Pro trial version www.pdffactory.com

PDF created with FinePrint pdfFactory Pro trial version www.pdffactory.com