* TM2033

Platform Technology

Parallel Computing

By Dr. Faizan Qamar

1
 THE JOURNEY

1 Computer 2 CPU, Memory 3 ISA 4 Computer

System System and
Performance
Input Output

5 Operating 6 Process 7 Memory and File 8 Operating

System Management Management System Design

9 Mobile Computing 10
Platform
Parallel Computing

2
 OUTLINE: PARALLEL COMPUTING
• INTRODUCTION
• PARALLEL COMPUTING
 Advantages Of Parallel
Computing
 Comparison Between Serial And
Parallel Computing
 Parallel Computing Architecture
 Flynn's Classification
• CLUSTER COMPUTING
 Why Cluster Computing ?
 Types Of Cluster Computing
 Advantages Of Cluster
Computing
 Disadvantages Of Cluster
Computing
• GRID COMPUTING
 Working Of Grid Computing
 Advantages Of Grid Computing
 Disadvantages Of Grid Computing
 CLUSTER Vs GRID COMPUTING
• CLOUD COMPUTING
 Types Of Cloud Computing
 Cloud Computing Services
 Advantages Of Grid Computing
 Disadvantages Of Grid Computing
3
INTRODUCTION: PARALLEL COMPUTING

4
DEFINATION OF PARALLEL COMPUTING
• Parallel computing is breaking up a Problem into
smaller pieces and executing those pieces at the same
time, each on their own processor or computer.
• Some examples of parallel computing include weather
forecasting, movie special effects, etc.

5
ADVANTAGES OF PARALLEL
COMPUTING
1. SAVE TIME AND/OR MONEY
In theory, throwing more resources at a task will shorten its time to
completion, with potential cost savings.
2. SOLVE LARGER / MORE COMPLEX PROBLEMS
Many problems are so large and/or complex that it is impractical or
impossible to solve them using a serial program, especially given limited
computer memory.
3. PROVIDE CONCURRENCY
Collaborative Networks provide a global venue where people from around
the world can meet and conduct work "virtually".
4. TAKE ADVANTAGE OF NON-LOCAL RESOURCES
Using computer resources on a wide area network, or even the Internet when
local computer resources are scarce or insufficient.

6
 COMPARISON BETWEEN SERIAL
AND PARALLEL COMPUTING

* Serial Computing * Parallel Computing

* Are uniprocessor systems ( 1 CPU) * Are Multiprocessor Systems ( many
* Can Execute 1 instruction at a time CPU’s )
* Speed is limited * Can Execute several Instructions at a
time.
* It is quite expensive to make single * No limitation on speed
CPU faster
* Area where it can be used : colleges, * Less expensive if we use larger
labs. number of fast processors to achieve
better performance.

7
 PARALLEL COMPUTING
ARCHITECUTRE
* There are a number of different ways to classify parallel computers.
* One of the more widely used classifications, in use since 1966, is called
Flynn's Taxonomy.
* Flynn's Classification:
* Flynn's taxonomy distinguishes multi-processor computer architectures
according to how they can be classified along the two independent dimensions
of Instruction Stream and Data Stream.
* Each of these dimensions can have only one of two possible states: Single or
Multiple.
* The sequence of instructions read from memory constitutes an instruction
stream.
* The operations performed on the data in the processor constitute a data
stream.

8
FLYNN'S CLASSIFICATION
Four major groups that are:
* Single instruction stream, single data (SISD)
* Single instruction stream, multiple data (SIMD)
* Multiple instruction stream, single data (MISD)
* Multiple instruction stream, multiple data (MIMD)

9
 Single instruction stream, single data
(SISD)
• A serial (non-parallel) computer
• Single Instruction: Only one instruction stream is being acted on by the CPU during
any one clock cycle
• Single Data: Only one data stream is being used as input during any one clock cycle
• Deterministic execution
• This is the oldest type of computer
• Examples: older generation mainframes, minicomputers, workstations and single
processor/core PCs.

10
 Single instruction stream, multiple
data (SIMD)
* A type of parallel computer
* Single Instruction: All processing units execute the same instruction at any given clock
cycle
* Multiple Data: Each processing unit can operate on a different data element
* Best suited for specialized problems characterized by a high degree of regularity, such
as graphics/image processing.
Examples:
* Most modern computers, particularly those with graphics processor units (GPUs)
employ SIMD instructions and execution units.

11
 Multiple instruction stream, single
data (MISD)
* A type of parallel computer
* Multiple Instruction: Each processing unit operates on the data independently via
separate instruction streams.
* Single Data: A single data stream is fed into multiple processing units.
* Multiple cryptography algorithms attempting to crack a single coded message.

12
 Multiple instruction stream, multiple
data (MIMD)
* A full type of parallel computer
* Multiple Instruction: Every processor may be executing a different instruction stream
* Multiple Data: Every processor may be working with a different data stream
* Execution can be synchronous or asynchronous, deterministic or non-deterministic
* Currently, the most common type of parallel computer - most modern supercomputers
fall into this category.

13
 PARALLEL PROGRAMMING
MODELS
There are several parallel programming models in common use:

* Shared Memory (without threads)

* Threads
* Distributed Memory / Message Passing
* Data Parallel
* Hybrid
* Single Program Multiple Data (SPMD)
* Multiple Program Multiple Data (MPMD)

14
CLUSTER COMPUTING
* Cluster computing is a collection of tightly or loosely connected computers that
work together so that they act as a single entity.
* The connected computers execute operations all together thus creating the idea
of a single system.
* The clusters are generally connected through fast local area networks (LANs).

15
TYPES OF CLUSTER COMPUTING
1. High performance (HP) clusters :
• HP clusters use computer clusters and supercomputers to solve advance computational
problems.
• They are used to performing functions that need nodes to communicate as they perform their
jobs.
• They are designed to take benefit of the parallel processing power of several nodes.

2. Load-balancing clusters :
• Incoming requests are distributed for resources among several nodes running similar programs
or having similar content.
• This prevents any single node from receiving a disproportionate amount of task.
• This type of distribution is generally used in a web-hosting environment.

3. High Availability (HA) Clusters :

• HA clusters are designed to maintain redundant nodes that can act as backup systems in case any
failure occurs.
• Consistent computing services like business activities, complicated databases, customer services
like e-websites and network file distribution are provided.
• They are designed to give uninterrupted data availability to the customers.
16
 ADVANTAGES OF CLUSTER
COMPUTING
1. High Performance :
The systems offer better and enhanced performance than that of mainframe computer
networks.
2. Easy to manage :
Cluster Computing is manageable and easy to implement.
3. Scalable :
Computer clusters can be expanded easily by adding additional computers to the network.
Cluster computing is capable of combining several additional resources or the networks to
the existing computer system.
4. Availability :
The other nodes will be active when one node gets failed and will function as a proxy for
the failed node. This makes sure for enhanced availability.
5. Flexibility :
It can be upgraded to the superior specification or additional nodes can be added.

17
DISADVANTAGES OF CLUSTER
COMPUTING
1. High cost :
It is not so much cost-effective due to its high hardware and its design.

2. Problem in finding fault :

It is difficult to find which component has a fault.

3. More space is needed :

Infrastructure may increase as more servers are needed to manage and monitor.

18
GRID COMPUTING
* Grid computing is a group of networked computers which work
together as a virtual supercomputer to perform large tasks, such as
analyzing huge sets of data or weather modeling.
* Grid computing is the use of widely distributed computer resources to
reach a common goal.
* It is a process architecture that combines different computing resources
from multiple locations to achieve desired and common goal.

19
WORKING OF GRID COMPUTING
Grid computing is distinguished from conventional high-performance computing
systems such as cluster computing in that grid computers have each node set to
perform a different task/application.
A Grid computing network combinedly works by using three types of computer:
Control Node:
A computer, usually a server or a group of servers which administrates the whole
network and keeps the account of the resources in the network pool.

Provider:
The computer which contributes it’s resources in the network resource pool.

User:
The computer that uses the resources on the network.

20
ADVANTAGES OF GRID COMPUTING

1) It is not centralized, as there are no servers required, except the control node
which is just used for controlling and not for processing.

2) Multiple heterogeneous machines i.e., machines with different Operating

Systems can use a single grid computing network.

3) Tasks can be performed parallelly across various physical locations and the
users don’t have to pay for it(with money).

21
DISADVANTAGES OF GRID
COMPUTING
1) Grid software and standards are still evolving

2) Complicated Architecture

3) Non-interactive job submission

4) High performance computer needed

5) Required delay-less internet connectivity

22
CLUSTER vs GRID COMPUTING

23
CLOUD COMPUTING

* Cloud computing is the delivery of on-demand computing services -- from

applications to storage and processing power -- typically over the internet and
on a pay-as-you-go basis.

* Rather than owning their own computing infrastructure or data centers,

companies can rent access to anything from applications to storage from a
cloud service provider.

24
TYPES OF CLOUD COMPUTING

There are 4 main types of cloud computing:

1) Private clouds
2) Public clouds
3) Hybrid clouds
4) Community clouds

25
CLOUD COMPUTING SERVICES

* Cloud computing services cover a vast range of options now, from the basics of
storage, networking, and processing power through to natural language
processing and artificial intelligence as well as standard office applications.

* Pretty much any service that doesn't require you to be physically close to the
computer hardware that you are using can now be delivered via the cloud.

Services:
• Servers
• Storage
• Databases
• Networking
• Softwares

26
ADVANTAGES OF CLOUD
COMPUTING

27
DISADVANTAGES OF CLOUD
COMPUTING
1) Internet connectivity
2) Vendor lock-in
3) Limited control
4) Data loss or theft
5) Data leakage
6) Account or service hijacking
7) Downtime

28
THANK YOU

29