Introduction to Distributed Systems

Distributed And Parallel

System

BY: M. TAHIR MUMTAZ

1
 Contents
1. Definition of Distributed System

2. Examples/Application of Distributed System

3. Advantage and Disadvantages of Distributed System

4. Definition Of Parallel System

5. Examples/Applications of Parallel System

6. Advantages and Disadvantages of Parallel System

7. References/Sources
DEFINITION
 A distributed system is a collection of independent
computers, interconnected via a network, capable of
collaborating on a task.
 A distributed system can be characterized as collection
of multiple autonomous computers that communicate
over a communication network and having following
features:
 No common Physical clock
 Enhanced Reliability
 Increased performance/cost ratio
 Access to geographically remote data and resources
 Scalability
3
 Distributed Systems
A collection of independent computers that
appear to the users of the system as a single
computer
A collection of autonomous computers,
connected through a network and distribution
middleware which enables computers to
coordinate their activities and to share the
resources of the system, so that users perceive
the system as a single, integrated computing
facility
A Distributed System

4
Examples of Distributed System

 Telephone Networks and Cellular Networks

 Computer Networks Such as internet or
intranet
 ATM(bank) Machines
 Distributed database and distributed database
management system
 Network of Workstations
 Mobile Computing etc.

5
6
Why Distributed Systems?
Design Requirements
 Performance issues
 Responsiveness
 Throughput

 Quality of service
 Correctness
 Reliability, availability, fault tolerance
 Security
 Performance
 Adaptability
Advantages Of Distributed System

 Information Sharing among Distributed Users

 Resource Sharing
 Extensibility and Incremental growth
 Shorter Response Time and Higher Output
 Higher Reliability
 Better Flexibility’s in meeting User’s needs
 Better price/performance ratio
 Scalability
 Transparency

7
Disadvantages of Distributed System

 Difficulties of developing distributed

software
 Networking Problem
 Security Problems
 Performance
 Openness
 Reliability and Fault Tolerance

8
 Basic problems and challenges
 Transparency
 Scalability
 Fault tolerance
 Concurrency
 Openness
 These challenges can also be seen as the goals or desired properties of a
distributed system
Transparency
 Concealment from the user and the application programmer of the
separation of the components of a distributed system
 Access Transparency - Local and remote resources are accessed in same way
 Location Transparency - Users are unaware of the location of resources
 Migration Transparency - Resources can migrate without name change
 Replication Transparency - Users are unaware of the existence of multiple
copies of resources
 Failure Transparency - Users are unaware of the failure of individual
components
 Concurrency Transparency - Users are unaware of sharing resources with
others
Scalability
 Addition of users and resources without suffering a noticeable loss
of performance or increase in administrative complexity

 Adding users and resources causes a system to grow:

 Size - growth with regards to the number of users or resources
 System may become overloaded
 May increase administration cost

 Geography - growth with regards to geography or the distance between nodes

 Greater communication delays

 Administration – increase in administrative cost

Openness
 Whether the system can be extended in various ways without troublesome
existing system and services
 Hardware extensions
 adding peripherals, memory, communication interfaces
 Software extensions
 Operating System features
 Communication protocols

 Openness is supported by:

 Public interfaces
 Standardized communication protocols
Concurrency
 In a single system several processes are interleaved
 In distributed systems - there are many systems with one or more
processors
 Many users simultaneously invoke commands or applications, access and
update shared data

 Mutual exclusion
 Synchronization
 No global clock
Fault tolerance
 Hardware, software and networks fail
 Distributed systems must maintain availability even at low levels of
hardware, software, network reliability
 Fault tolerance is achieved by
 Recovery

 Redundancy

 Issues
 Detecting failures
 Recovery from failures
 Redundancy
Fault tolerance
Omission and Arbitrary Failures
Parallel System

 A system is said to be a Parallel System in which multiple

processor have direct access to shared memory which forms
a common address space.
 Usually tightly-coupled system are referred to as Parallel
System. In these systems, there is a single system wide
primary memory (address space) that is shared by all the
processors. On the other hand Distributed System are
loosely-coupled system.
 Parallel computing is the use of two or more processors
(cores, computers) in combination to solve a single
problem.

9
Examples
Loosely-Coupled Systems

 Most distributed systems are “loosely-coupled:

 Each CPU runs an independent autonomous OS.
 Hosts communicate through message passing.
 Computer don’t really trust each other.
 Some resources are shared, but most are not.
 The system may look differently from different
hosts.
 Typically, communication times are long.
Tightly-Coupled Systems

 A “tightly-coupled” system usually refers to a multiprocessor.

 Runs a single copy of the OS with a single job queue
 has a single address space
 usually has a single bus or backplane to which all
processors and memories are connected
 processors communicate through shared memory
A Parallel System

10
Applications of Parallel System

 An example of Parallel computing would

be two servers that share the workload of
routing mail, managing connections to an
accounting system or database, solving a
mathematical problem etc
 Supercomputers are usually placed in
parallel system architecture
 Terminals connected to single server

11
EXAMPLE: The Earth Simulator Supercomputer
from (2002-2004)

12
Advantages of Parallel System

 Provide Concurrency(do multiple things at the

same time)
 Taking advantage of non-local resources
 Cost Savings
 Overcoming memory constraints
 Save time and money
 Global address space provides a user-friendly
programming perspective to memory

13
Disadvantages of Parallel System

 Primary disadvantage is the lack of scalability

between memory and CPUs.
 Programmer responsibility for
synchronization constructs that ensure
"correct" access of global memory.
 It becomes increasingly difficult and
expensive to design and produce shared
memory machines with ever increasing
numbers of processors.

14
 Parallel vs. Distributed System

Parallel Systems Distributed Systems

Memory Tightly coupled system Weakly coupled system

shared memory Distributed memory

Control Global clock control No global clock control

Processor Order of Tbps Order of Gbps

interconnection

Main focus Performance Performance(cost and scalability)

Scientific computing Reliability/availability
Information/resource sharing

15
Sources / References

Websites  en.wikipedia.org
 books.google.com
 www.seminarprojects.com
 http://publib.boulder.ibm.com
 www.webopedia.com
 https://computing.llnl.gov/tutorials
 www.cis.upenn.edu

Books  Distributed System by Coulouris

 Distributed Computing by D. Kshemkalyani,Mukesh Singhal

16
Thank
You