Cloud Computing

Saswati Mukherjee
 What Is Service?

• Service is what you connect together using Web Services.

• Service is the endpoint of a connection.
• Functionalities of service :
• A service should be well-defined
• A service should be self-contained
• A service should not depend on the context or state of other services.
 What Is Web Service?

• Definition :
• Web service is self-describing and stateless modules that perform discrete
units of work and are available over the network
• Web service providers offer APIs that enable developers to exploit
functionality over the Internet, rather than delivering full-blown applications
 Service Oriented Architecture
• Definition
• Service Oriented Architecture (SOA) is essentially a
collection of services which communicate with each other
• Contain a flexible set of design principles used during the
phases of systems development and integration
• Provide a loosely-integrated suite of services that can be
used within multiple business domains
• Approach
• Usually implemented by Web Service model
 Cloud: Client Server

• Cloud applications are based on

distributed client-server paradigm
– Servers run in the providers’ premise
• the computations are carried out on the
cloud.
• Data resides in the cloud
– A thin client runs on the user’s
machine
 Cloud: Distributed Systems

• Clouds are essentially large

distributed systems that make
available their services to third
parties on demand.
• So, how is a distributed system
related to distributed computing?
 Distributed Systems

• A distributed system is a software system in which

components located on networked
computers communicate and coordinate their actions
by passing messages. 
• Distributed computing is a field of computer science that
studies distributed systems.
 Distributed Systems

• Design of an information system

• Layers and tiers
• Bottom up design
• Top down design
• Architecture of an information system
• One tier
• Two tier (client/server)
• Three tier (middleware)
• N-tier architectures
• Clusters and tier distribution
• Communication in an information system
• Blocking or synchronous interactions
• Non-blocking or asynchronous interactions
 Layers and tiers

Client Presentation layer • Client is any user or program that wants to perform an
operation over the system. Clients interact with the
system through a presentation layer
Application Logic Business rules

• The application logic determines what the system

Resource Manager Business objects actually does. It takes care of enforcing the business rules
and establish the business processes. The application
logic can take many forms: programs, constraints,
business processes, etc.

Client Client
• The resource manager deals with the organization
(storage, indexing, and retrieval) of the data necessary to
Server Business processes support the application logic. This is typically a database
but it can also be a text retrieval system or any other
data management system providing querying capabilities
Database Persistent storage and persistence.
 Boxes and arrows
There is no problem in system
design that cannot be solved
by adding a level of indirection.
There is no performance
problem that cannot be solved
by removing a level of
indirection.
• Each box represents a part of the system.
• Each arrow represents a connection between two parts
of the system.
• The more boxes, the more modular the system: more
opportunities for distribution and parallelism. This allows
encapsulation, component based design, reuse.
• The more boxes, the more arrows: more sessions
(connections) need to be maintained, more coordination
is necessary. The system becomes more complex to
monitor and manage.
• The more boxes, the greater the number of context
switches and intermediate steps to go through before
one gets to the data. Performance suffers considerably.
• System designers try to balance the flexibility of modular
design with the performance demands of real
applications. Once a layer is established, it tends to
migrate down and merge with lower layers.
 Top down design
• The functionality of a system is divided among
several modules. Modules cannot act as a top-down architecture
separate component, their functionality
depends on the functionality of other modules.
PL-B
• Hardware is typically homogeneous and the PL-A
system is designed to be distributed from the
beginning.
PL-C

top-down design

PL-A PL-B AL-B

L-C
PL-C A

D
L-
A
AL-B -C
D AL-A
L
L-

AL-A A
A

RM-1 RM-2 RM-1 RM-2

 Top down design

top-down design

1. define access channels

and client platforms client
2. define presentation
formats and protocols for
the selected clients and presentation
layer

information system
protocols

3. define the functionality

necessary to deliver the application logic
contents and formats needed layer
at the presentation layer

4. define the data sources resource management

and data organization needed layer
to implement the application
logic
 Bottom up design
• In a bottom up design, many of the basic
New Legacy
components already exist. These are stand
application application alone systems which need to be integrated
into new systems.
• The components do not necessarily cease to
work as stand alone components. Often old
applications continue running at the same
time as new applications.
• This approach has a wide application because
the underlying systems already exist and
cannot be easily replaced.
• Much of the work and products in this area
are related to middleware, the intermediate
layer used to provide a common interface,
bridge heterogeneity, and cope with
Legacy systems
distribution.
 Bottom up design

bottom-up design PL-B

PL-A

bottom-up architecture
PL-A PL-B
PL-C PL-C

AL-B
-C AL-B

D
L -C

L-
AL-A A AL

D
L-
wrapper wrapper wrapper

A
AL-A

wrapper wrapper wrapper

legacy legacy legacy legacy legacy

application application system system system
 Bottom up design

bottom-up design

1. define access channels

and client platforms client

2. examine existing resources

and the functionality presentation
they offer layer

information system
3. wrap existing resources application logic
and integrate their functionality layer
into a consistent interface

4. adapt the output of the resource management

application logic so that it layer
can be used with the required
access channels and client
protocols
 One tier: fully centralized
1-tier architecture • The presentation layer, application logic
and resource manager are built as a
monolithic entity.
• Users/programs access the system through
display terminals but what is displayed and
how it appears is controlled by the server.
(These are “dumb” terminals).
Server • This was the typical architecture of
mainframes, offering several advantages:
• no forced context switches in the control flow
(everything happens within the system),
• all is centralized, managing and controlling
resources is easier,
• the design can be highly optimized by blurring
the separation between layers.
 Two tier: client/server
• As computers became more powerful, it was possible to
2-tier architecture move the presentation layer to the client. This has
several advantages:
• Clients are independent of each other: one could
have several presentation layers depending on what
each client wants to do.
• One can take advantage of the computing power at
the client machine to have more sophisticated
Server presentation layers. This also saves computer
resources at the server machine.
• It introduces the concept of API (Application
Program Interface). An interface to invoke the
system from the outside. It also allows designers to
think about federating the systems into a single
system.
• The resource manager only sees one client: the
application logic. This greatly helps with
performance since there are no client
connections/sessions to maintain.
 API in client/server
• Client/server systems introduced the notion of service (the client invokes a service implemented by the server)
• Together with the notion of service, client/server introduced the notion of service interface (how the client can
invoke a given service)
• Taken all together, the interfaces to all the services provided by a server (whether there are application or system
specific) define the server’s Application Program Interface (API) that describes how to interact with the server from
the outside
• Many standardization efforts were triggered by the need to agree to common APIs for each type of server

server’s API

service service service service

interface interface interface interface

server
service service service service

resource management
layer
 Thin Client

• What is thin client ?

• Thin client is a computer or a computer program which depends heavily on some
other computer to fulfill its traditional computational roles. This stands in contrast
to the traditional fat client, a computer designed to take on these roles by itself.
• Characteristics :
• Cheap client hardware
• While the cloud providers handle several client sessions at once, the clients can be made out
of much cheaper hardware.
• Diversity of end devices
• End user can access cloud service via plenty of various electronic devices, which include
mobile phones and smart TV.
• Client simplicity
• Client local system do not need complete operational functionalities.
 Technical aspects of the 2 tier architecture

• There are clear technical advantages when going from one tier to two tier architectures:
• take advantage of client capacity to off-load work to the clients
• work within the server takes place within one scope (almost as in 1 tier),
• the server design is still tightly coupled and can be optimized by ignoring presentation issues
• still relatively easy to manage and control from a software engineering point of view

• However, two tier systems have disadvantages:

• The server has to deal with all possible client connections. The maximum number of clients is given by
the number of connections supported by the server.
• Clients are “tied” to the system since there is no standard presentation layer. If one wants to connect
to two systems, then the client needs two presentation layers.
• There is no failure or load encapsulation. If the server fails, nobody can work. Similarly, the load
created by a client will directly affect the work of others since they are all competing for the same
resources.
 The main limitation of client/server
Server A Server B
 If clients want to access two or more
servers, a 2-tier architecture causes
several problems:
 the underlying systems don’t
know about each other
 there is no common business
logic
 the client is the point of
integration (increasingly fat
clients)
• The responsibility of dealing with
heterogeneous systems is shifted to the
client.
• The client becomes responsible for
knowing where things are, how to get
to them, and how to ensure consistency
• This is tremendously inefficient from
all points of view (software design,
portability, code reuse, performance
since the client capacity is limited,
etc.).
• There is very little that can be done
to solve this problems if staying
within the 2 tier model.
 Three tier: middleware

3-tier architecture
• In a 3 tier system, the three layers are
fully separated.
• The layers are also typically distributed
taking advantage of the complete
modularity of the design (in two tier
systems, the server is typically
centralized)
• A middleware based system is a 3 tier
architecture. This is a bit oversimplified
but conceptually correct since the
underlying systems can be treated as
black boxes. In fact, 3 tier makes only
sense in the context of middleware
systems (otherwise the client has the
same problems as in a 2 tier system).
 Middleware

• Middleware is just a level of indirection

clients between clients and other layers of the
Middleware or system.
global application logic • It introduces an additional layer of business
Local application logic
logic encompassing all underlying systems.
• By doing this, a middleware system:
Local resource • simplifies the design of the clients by reducing
managers the number of interfaces,
• provides transparent access to the underlying
systems,
• acts as the platform for inter-system functionality
middleware and high level application logic, and
• takes care of locating resources, accessing them,
and gathering results.
• But a middleware system is just a system like
any other! It can also be 1 tier, 2 tier, 3 tier ...
Server A Server B
 Technical aspects of middleware

• The introduction of a middleware layer helps in that:

• the number of necessary interfaces is greatly reduced:
• clients see only one system (the middleware),
• local applications see only one system (the middleware),
• it centralizes control (middleware systems themselves are usually 2 tier),
• it makes necessary functionality widely available to all clients,
• it allows to implement functionality that otherwise would be very difficult to provide, and
• it is a first step towards dealing with application heterogeneity (some forms of it).
• The middleware layer does not help in that:
• it is another indirection level,
• it is complex software,
• it is a development platform, not a complete system
A three tier middleware based system ...
External clients
External client
middleware system
internal
clients control

connecting logic

user logic middleware

wrappers
2 tier systems

Resource
managers 2 tier system Resource
manager
 N-tier: connecting to the Web

client • N-tier architectures result from

N-tier connecting several three tier systems
Web browser architecture to each other and/or by adding an
additional layer to allow clients to
access the system through a Web
server
Web server
• The Web layer was initially external
presentation to the system (a true additional
layer layer); today, it is slowly being
HTML filter incorporated into a presentation
layer that resides on the server side
(part of the middleware
infrastructure in a three tier system,
or part of the server directly in a two
application logic tier system)
layer middleware • The addition of the Web layer led to
the notion of “application servers”,
which was used to refer to
resource management middleware platforms supporting
layer access through the Web
information system
N-tier systems in reality
INTERNET

FIREWALL
internal
clients Web
server
LAN cluster

LAN
middleware
application LAN,
logic gateways
LAN

resource middleware
application
Wrappers
and
management logic gateways
LAN LAN
layer
database file application additional resource
server server management layers
 Distributed System

• Definition
A distributed system is a collection of
autonomous computers linked by a
computer network that appear to the
users of the system as a single
computer.

Source: Coulouris, Dollimore and Kindberg

 Benefits of Distributed Systems
• Resource sharing
– Across boundaries of
computers
• Reliability
– High availability owing to
multiplicity of resources
• Computation speed-up
– Processes of an application
can be scheduled
simultaneously
• Communication
– Users and processes in
different nodes can
communicate
• Incremental growth
– Cost of enhancing a capability
is α additional capability
desired
• Made possible by open system
standards
 Design Challenges

➢ Heterogeneity ➢ Security ➢ Failure handling

➢ Heterogeneous ➢ system should only be ➢ Failure of a component
components must be used in the way should not result in
able to interoperate. intended. failure of the whole
system.
➢ Openness ➢ Scalability
➢ Concurrency
➢ Interfaces should ➢ System should work
➢ Services and
allow components to efficiently with an
increasing number of applications can be
be added or
shared by clients in a
replaced. users.
distributed system.
➢ Availability ➢ System performance
➢ Transparency
➢ The systems would should increase with
inclusion of additional ➢ Distribution should be
always be available. hidden from the user
resources.
as much as possible.
 Transparency

Transparency Description Transparenc Description

y
Access Hide differences in data Concurrency Hide the possibility that the
representation & resource access resource may be shared
(enables interoperability) concurrently
Location Hide location of resource (can Failure Hide failure and recovery of
use resource without knowing its the resource. How does one
location) differentiate between slow
Migration Hide possibility that a system and failed?
may change location of resource Relocation Hide that resource may be
moved during use
Replication Hide the possibility that
multiple copies of the
resource exist
 Distributed Systems: Cloud

• The cloud system have particularly

inherited the following distributed
characteristics:
– Continuous availability,
– Scalability
– Concurrency
 Communication

• Interaction of two processes

– Or two or more entities
• Uniprocessor system – easy
– Existence of shared memory.
•  A typical example is the producer-consumer problem.
– One process writes to a buffer
– Read by another process
• Synchronization required?
• Uses semaphore.
.
 Communication

• Distributed system – hard

– No shared memory.
• Processes are located in geographically dispersed
locations.
• All communication in distributed system is based on
message passing.
.
 Communication

• Interaction of two processes

– Or two or more entities
• Uniprocessor system – easy
– Existence of shared memory.
•  A typical example is the producer-
consumer problem.
– One process writes to a buffer
– Read by another process
• Synchronization required?
.• Uses semaphore.
 Communication

• Distributed system – hard

– No shared memory.
• Processes are located in
geographically dispersed locations.
• All communication in distributed
system is based on message
passing.
.
 Message Passing Example

• Process A wants to communicate

with Process B
– A first builds a message in its own
address space
– It executes a system call
– The OS of A sends the message through
. network to B.
.
 Who Owns the Internet?

• No single person or company owns the

Internet or even controls it entirely.
• As a wide-area network, it is made up of
many smaller networks.
• These smaller networks are often
owned and managed by a person or
organization.
• The Internet is really defined by how
. connections can be made between
these networks.
 Relationship with Cloud
• Cloud computing is closely related to the
Internet
– Clients of the clouds are connected to the CSPs’
(Cloud Service Providers’) servers through the
Internet
• CSP-s servers are located in different data
centers which are geographically dispersed.
• Data of a client may be located in a server in
one data center and a computation on that data
may be carried out in another server located in
another data center.
. • The two servers need to be connected and this
is done using the Internet
 TCP/IP Suite

Application Application

Transport TCP,. UDP Transport

Network IP Network

Data Link Data Link

Physical Link
 Reliable Communication
• TCP: Reliable, connection oriented
• To satisfy the integrity property of reliable
communication,
– TCP uses checksums to detect and reject corrupt
packets and
– Uses sequence numbers to detect and reject duplicate
packets.
• For the sake of the validity property, TCP uses
timeouts and retransmissions to deal with lost
packets.
• Hence, messages are guaranteed to be delivered
even when some of the underlying packets are lost.
. • UDP, Unreliable and connectionless.
 Distributed Communication

• Distributed system is a very large and

complex environment
• To understand the fundamental building
blocks of a distributed system, it is
necessary to consider two key questions:
• What are the entities that can communicate
• How do they communicate, or, more
specifically, what communication paradigm
. is used?
 Entities: Objects

• Objects: Objects in distributed object-

based environment.
• Typically, objects are accessed via
interfaces, with an associated
interface definition language (or IDL).
•. They require specific method of
.
communicating with each other.
 Entities: Components

• Components, in a large distributed

environment like cloud, offer problem-
oriented abstractions.
• Components have clearly defined functions
• Components clearly defined interfaces.
• Components also specify the assumptions
in terms of other components/interfaces
that must be present in the distributed
system.
. •. Therefore, all dependencies are explicit..
 Entities: Web Services
• Web services: Web services represent a very
important paradigm for the development of
distributed systems, especially cloud systems.
• Web services also take the approach based on
encapsulation of behaviour and access through
interfaces.
• The World Wide Web consortium (W3C) defines a
web service as:
A software application identified by a URI, whose interfaces
and bindings are capable of being defined, described and
discovered as XML artefacts. A Web service supports
direct interactions with other software agents using XML-
based message exchanges via Internet-based protocols.
. .
 Distributed Communication Paradigm

Three types of communication

paradigm:
•interprocess communication;
•remote invocation;
. •indirect communication.
 Paradigm 1: Interprocess communication

• Interprocess communication refers to

the relatively low-level support for
communication between processes in
distributed systems.
• It includes
– message-passing primitives,
– direct access to the API offered by Internet
protocols (socket programming) and support
for
. – multicast communication.
 Interprocess communication

• IPC is a fundamental aspect of

distributed systems design and
implementation.
• Two purposes
– Either exchange data and information or
– Coordinate the activity of processes.
• IPC is what ties together the different
. components of a distributed system,
thus making them act as a single
system.
 Paradigm 2: Remote Invocation

• Remote invocation represents the most

common communication paradigm in
distributed systems,
• Covers a range of techniques based on
a two-way exchange between
communicating entities in a distributed
system and
• Results in the calling of a remote
operation, procedure or method.
. • Can take many forms.
 Remote Invocation
Request Reply
• Request-reply protocols are effectively a
pattern imposed on an underlying
message-passing service to support
client-server computing.
• Typically involve a pairwise exchange of
messages from client to server and
back
• Rather primitive paradigm
. • Typically used in embedded systems.
 Remote Invocation
Remote Procedure Call
• An elegant communication mechanism
attributed to Birrell and Nelson,1984.
• In RPC, procedures in processes on
remote computers can be called as if
they are procedures in the local.
• A client-server communication
mechanism:
– servers offer a set of operations through a
service interface
– clients call these operations directly as if
. they are available locally.
 RPC

The underlying RPC system then hides

many complex aspects:
•Complexity of distribution,
•Complexity of encoding and decoding
of parameters and results,
•Complexity of passing of messages
. while preserving the required
semantics for the procedure call.
 RPC Steps
1. Client procedure calls client stub in normal
way
2. Client stub builds message, calls local OS
3. Client's OS sends message to remote OS
4. Remote OS gives message to server stub
5. Server stub unpacks parameters, calls server
6. Server does work, returns result to the stub
7. Server stub packs it in message, calls local
OS
8. Server's OS sends message to client's OS
9. Client's OS gives message to client stub
. 10. Stub unpacks result, returns to client
RPC Steps
 Reliable RPC

• What transport layer protocol to be

used in RPC to ensure reliability?
• TCP
• The reliability mechanisms built into
. the TCP protocol subsume the need
for the RPC protocol to implement
any form of acknowledgement or
retransmission policy of its own.
 Remote Invocation
Remote Method Invocation

• Remote method invocation (RMI)

strongly resembles remote procedure
calls but in a world of distributed
objects.
• With this approach, a calling object can
invoke a method in a remote object.
• As with RPC, the underlying details are
.
generally hidden from the user.
 Paradigm 3: Group Communication

• Group communication is concerned with

the delivery of messages to a set of
recipients and hence is a multiparty
communication paradigm supporting
one-to-many communication.
• Group communication relies on the
. abstraction of a group which is
represented in the system by a group
identifier.
 Group Communication

• Reliable communication
• Considerations
– Types of groups
– Types of communication in group
communication
 Reliable Group Communication
• A group communication service is reliable if the
communication service is able to mask some of the
failures that would be injected by the unreliable
channel.
• Definition: A group communication is reliable if it
possesses validity, integrity and agreement:
(applicable for all distributed communication)
– Validity: A message sent by a sender is eventually
received by the receiver.
– Integrity: two assurances
• The message received is identical to one sent,
• No messages are delivered twice: implies at most once protocol.
– Agreement: If a sent message is delivered to one of the
processes in a group, it is delivered to all processes in
the group.
• Applicable only in a group communication.
 Validity: Problem

• Communication using a message

• Communication over the network
• Network is unreliable
– Packets are dropped
– Congestion
– Network partitioning
 Integrity: Problem

The threats to integrity come from two

independent sources:
•Any protocol that retransmits messages but does
not reject a message that arrives twice. Solution?
– Protocols can attach sequence numbers to
messages so as to detect those that are delivered
twice.
•Malicious users that may inject spurious
messages, replay old messages or tamper with
messages. Solution?
– Security measures can be taken to maintain the
integrity property in the face of such attacks.
 Agreement: Problem

• Essentially, in a group of
communicating nodes, a cooperative
agreement is difficult!
• Network failures.
 Types of Groups

• Closed Group • Synchronous Systems

• Open Group • Asynchronous Systems
• Overlapping Group
• Non-overlapping Group

•What groups are there in cloud?

 Blocking or synchronous interaction
client server
Call
Receive

• Traditionally, information systems use blocking idle timecalls (the client sends a

request to a service and waits for a Answer

response of the service to come back
Response

before continuing doing its work)

• Synchronous interaction requires both• Because
partiesitto be “on-line”:
synchronizes client theand caller
makes a request, the receiver gets the server,
request, this processes
mode of operation has
the request,
several disadvantages:
sends a response, the caller receives the response.
• connection overhead
• The caller must wait until the response comes back. The
• higher probability receiver does not
of failures
• difficult to identify
need to exist at the time of the call (TP-Monitors, CORBA and react
or to
DCOM create
failures
an instance of the service/server /object• when called system;
it is a one-to-one if it does not exist
it is not
already) but the interaction requires bothreallyclient andforserver
practical to be
nested calls and “alive”
complex interactions (the problems
at the same time becomes even more acute)
 Overhead of synchronism
• Synchronous invocations require to maintain a
session between the caller and the receiver.
• Maintaining sessions is expensive and consumes request()
CPU resources. There is also a limit on how many receive
sessions can be active at the same time (thus session
limiting the number of concurrent clients process
duration
connected to a server) return
• For this reason, client/server systems often resort do with answer
to connection pooling to optimize resource
utilization
request()
• have a pool of open connections
• associate a thread with each connection receive
• allocate connections as needed process
return

• Synchronous interaction requires a context for each do with answer

call and a context management system for all
incoming calls. The context needs to be passed
around with each call as it identifies the session,
the client, and the nature of the interaction.
Context is lost
Needs to be restarted!!
 Failures in synchronous calls
• If the client or the server fail, the context is 1
lost and resynchronization might be difficult. request()
• If the failure occurred before 1, nothing 2
receive
has happened process
• If the failure occurs after 1 but before 2 return
(receiver crashes), then the request is 4 3
lost do with answer
• If the failure happens after 2 but before
3, side effects may cause inconsistencies
• If the failure occurs after 3 but before 4, 1
the response is lost but the action has request()
been performed (do it again?) 2
receive
• Who is responsible for finding out what process
happened? return
• Finding out when the failure took place may 3
not be easy. Worse still, if there is a chain of do with answer
invocations (e.g., a client calls a server that timeout
calls another server) the failure can occur 2’
anywhere along the chain. try again receive
process
do with answer return
3’
 Two solutions

ASYNCHRONOUS
ENHANCED SUPPORTINTERACTION
• Using
Client/Server
asynchronous
systemsinteraction,
and middleware
the caller
platforms
sends provide
a message
a number
that gets
of
mechanisms
stored somewhere
to dealuntil
withthe
thereceiver
problems reads
created
it andbysends
synchronous
a response.
interaction:
The response is sent in a similar manner
• Transactional interaction
• Asynchronous interaction: to enforce
can take exactly
place inonce
two execution
forms: semantics and
enable more complex interactions with some execution guarantees
• non-blocking invocation (a service invocation but the call returns immediately
• Service
withoutreplication
waiting forand load balancing:
a response, to batch
similar to prevent the service from becoming
jobs)
unavailable when there is a failure (however, the recovery at the client side is
• persistent queues (the call and the response are actually persistently stored
still a problem of the client)
until they are accessed by the client and the server)
 Message queuing

• Reliable queuing turned out to be a very good idea and an excellent

complement to synchronous interactions:
request()
• Suitable to modular design: the code for making a request queuecan be in a
receive
different module (even a different machine!) than the code for dealing with
process
the response return
• It is easier to design sophisticated distribution modes (multicast,
queue transfers,
replication, coalescing messages) an it alsodo with
helpsanswer
to handle communication
sessions in a more abstract way
• More natural way to implement complex interactions between
heterogeneous systems
Group Communication Mechanism

• Multicast: message sent to a group

of processes
• Broadcast: message sent to all
processes (anywhere)
• Unicast: message sent from one
sender process to one receiver
process
 Unicast vs. Multicast

*
 Boradcasting

P1 P2

P3
 Multicast

• Whereas the majority of network

services and network applications
use unicast for IPC, multicasting is
useful for many applications.
 Multicast Group
• In a service which uses multicasting, a set of
processes form a group, called a multicast
group.
• Each process in a group can send and
receive message.
• A message sent by any process in the group
can be received by each participating
process in the group.
• A new process may join a multicast group
and an existing process may also choose to
leave a multicast group.
 Multicasting

• When a multicast message is sent by

a process, the multicast mechanism
is responsible for delivering the
message correctly.
• Let’s see what is correctness here?
 Multicasting

• What is correctness?
– All nodes of the group should receive
the sent message.
– The complete message should be
received by all
– Messages should be delivered in order
 Correctness: All Nodes Receive

message m1 P3

P1
Multicasts
message m1 message m1
P4 message m1 P2
to a multicast
group having
7 members

message m1 P5
P7

• Why does it happen? P6

– Failures of network hosts
– Failures of network links
– Packets dropped
 Multicasting

• What is correctness?
– All nodes of the group should receive
the message
– The complete message should be
received by all
– Messages should be delivered in order
 Correctness: Complete Message

message 64 P3
bit m1
P1
Multicasts 64
bit message message 32
P2 message 64 P4
m1 to a
bit m1 bit m1’
multicast
group having
7 members
message 64 P5
message 64 P7 bit m1
bit m1
• Why does it happen? message 32 P6
bit m1’
– M1may consist of
two packets
– Some packets are dropped
 Multicasting

• What is correctness?
– All nodes of the group should receive
the message
– The complete message should be
received by all
– Messages should be delivered in order
 Correctness: Delivered in Order

message m1,
then m2

Multicasts two
message m1 message m1,
message m2
and m2 to a then m2
then m 1
multicast
group having
7 members
message m1
message m1,
then m2
then m2
message m2,
then m1

• Why does it happen?

– Routing delays
 Multicasting
• What is correctness?
– All nodes of the group should receive the message
– The complete message should be received by all
– Messages should be delivered in order
• Why is it not guaranteed?
– Failures of network hosts
– Failures of network links
– Routing delays
– Heterogeneity of nodes
– The distance from the sender node would vary
among the recipient processes.
• Is correctness required always?
 Correctness

• Is not a must always!

• There are some applications which can
tolerate an occasional miss or
misordering of messages
– video conferencing
• But there are applications for which
such anomalies are unacceptable
– database applications.
• If needed, who ensures correctness?
 Ordering of Messages

• The popular flavors implemented by several

multicast protocols as follows:
1. Unordered
2. FIFO ordering
3. Causal ordering
4. Total ordering
Parallel Computing
• Single-instruction, single-data (SISD) systems
• Single-instruction, multiple-data (SIMD) systems
• Multiple-instruction, single-data (MISD) systems
• Multiple-instruction, multiple-data (MIMD) systems
Parallel Computing
Parallel Computing
• A sequential program is one that runs on a single processor and has a
single line of control. To make many processors collectively work on a
single program, the program must be divided into smaller independent
chunks so that each processor can work on separate chunks of the
problem. The program decomposed in this way is a parallel program.
• A wide variety of parallel programming approaches are available. The
most prominent among them are the following:
• Data parallelism
• Process parallelism
• Farmer-and-worker model
Parallel Computing: Levels of Parallelism

Grain Size Code Item Parallelized By

Large
Medium Separate and heavyweight process
Programmer
Fine Function or procedure
Programmer
Very fine Loop or instruction block
Parallelizing compiler
Instruction
Processor
 eed

           
Number processors versus speed.
Cost versus speed.
Parallel Computing
• The development of parallel computing, which encompasses techniques, architectures, and
systems for performing multiple activities in parallel are referred to as Parallel Computing.
• The term parallel computing has blurred its edges with the term distributed computing and
is often used in place of the latter term.
• Processing of multiple tasks simultaneously on multiple processors is called parallel
processing.
• The parallel program consists of multiple active processes (tasks) simultaneously solving a
given problem.
• A given task is divided into multiple subtasks using a divide-and-conquer technique, and
each subtask is processed on a different central processing unit (CPU).
• Programming on a multiprocessor system using the divide-and-conquer technique is called
parallel programming.
Parallel Computing
• Parallel Processing
• Parallel processing is a form of computation in which many
calculations are carried out simultaneously, operating on
the principle that large problems can often be divided into
smaller ones, which are then solved concurrently.

• Parallelism in different levels :

• Bit level parallelism
• Instruction level parallelism
• Data level parallelism
• Task level parallelism
Parallel Computing
• Hardware approaches
• Multi-core computer
• Symmetric multi-processor
• General purpose graphic processing unit
• Vector processor
• Distributed computing
• Cluster computing
• Grid computing
• Software approaches
• Parallel programming language
• Automatic parallelization
Thank You!