Introduction

1
Parallel Computing
2
CO
N

3 Flynn’s Classification
TEN
TS

SISD, SIMD, MISD MIMD

4
1 INTRODUCTION
 What is parallel computing ?
It is a form of computer in which many calculations
are caried out simultaneously.

 Same thing or different things.

 Solving one larger problem.

To be run using multiple CPUs.

 Serial Computing
Traditionally, software has been written for serial computation:

 To be run on a single computer having a single CPU.

 A problem is broken into a discrete series of instructions

 Instructions are executed sequentially one after anoter executed on a

single processor

 Only one instruction may execute at any moment in time

 Parallel Computing
Parallel computing is a form of computation in which many calculations
are carried out simultaneously.

In the simplest sense, It is the simultaneous use of multiple compute

resources to solve a computational problem:

 To be run using multiple CPUs.

 A problem is broken into discrete parts that can be solved
concurrently.
 Each part is further broken down to a series of instructions.
 Instructions from each part execute simultaneously on different
CPUs
 Why Use Parallel Computing?
 Save time and money.
 Solve larger problem : eg- web search engines/databases processing
millions of transection per second.
 Provide concurrency.
 Use of non-local recources:

Limits of serial computing:

1. Tranmission speed - The speed of a serial computer is directly
dependent upon how fast data can move through hardware, transmission
limit of copper wire.(9cm/nanosecond)
2. Limits ro miniaturization
3. Economic limitations - it is increasigly to make a single processor faster
Current computer architectures are increasingly relying upon hardware
level parallelism to improve performance:

- Multiple execution units

- Piplined instructions

- Multi-core
 Where is Parallel Computing used ?

 Scientific Computing
Numerically Intensive Simulations
 Data Base Operations and Information System
Web based Business services, Web search Engines, Online Transection
Processing
 Data mining, Geographic Information system

 Artifical Intelligence , Machine Learning and Deep Learning

 Real Time Systems and Control Applications

2 Fylnn’s Classifications
 ARCHITECTURAL CLASSIFICATION SCHEMES
Three Computer Architectural Classification Schemes are:

1. Flynn’s Classification (1966):

It is based on the multiplicity of instruction stream and data streams
in a computer system.
SISD, SIMD, MISD, MIMD

2. Feng’s Classification (1972):

It is based on serial versus parallel processing.
WSBS, WPBS, WSBP, WPBS

3. Handler’s Classification (1977):

It is determinded by the degree of parallelism and pipelining in
various subsystems levels
 FLYNN’S CLASSIFICATION
Flynn’s Classification is the most popular taxonomy of computer architecture,
Proposed by MICHEAL J. FLYNN in 1966 based on number of instructions and
data.

It is based on the notion of a stream of information.

The term ‘stream’ refers to a sequence or flow of either instructions or data

operated on by the computer.

Two types of information flow into a processor : Instructions and data.

The Instruction Stream is defined as the sequence of instructions
executed by the processing unit.

The Data Stream is defined as the sequence of data including inputs,

partical, or temporary results, called by the instruction stream
 Types of Flynn’s Taxonomy

According to the flynn’s classification, either of the

instruction or data stream can be single or multiple.

The Computer architecture can be classified into four

categories : -
 SISD
01 Single Instruction -Single Data Stream

SIMD
02 Single Instruction - Multiple Data Stream

MISD
03 Multiple Instruction - Single Data Stream

MIMD
04 Multiple Instruction - Multiple Data Stream
3 Explanation of
Flynn’s Taxonomy
SISD- Single Instruction Stream - Single Data Stream
An SISD computing system is a uni-processor machine which is capable of
executing a single instruction operating on a single data stream.

 Single Instruction : only one instruction stream is being acted on by the

CPU during any one clock cycle.
 Single data : only one data stream is being used as input during any one
clock cycle.

convenitional single - processor Von Neumann Computer are classified as SISD

systems. It is a serial (non-parallel) computer.
 Instructions are executed sequentially, but may be overlapped in
their execution stages ( Pipelinning).

Most SISD uni - processor system are piplined

SISD computer may have more than one functional units, all
under the supervision of control unit.

EXAMPLES : Most PC’s , single CPU workstation, mini computers,

mainframes. CDC-6600, VAX 11, IBM 7001 are SISD computers.
SIMD - Single Instruction Stream - Multiple Data Stream
An SIMD system is capable of executing the same instruction on all the
CPU’s but operating on diffrent data streams.

 Single Instruction: All processing unit execute the same

instruction at any given clock cycle

 Multiple data : Each processing unit can operate on a different

data element
 SIMD computer has single control unit which issues one
instruction at a time but it has multiple ALU’s or processing units to
carry out on multiple data sets simultaneously

Well stuied to scientific computing since they involve lots of

vector and matrix operations.

Examples : Array Processor sand vector pipelines

Array processors : ILLIAC-IV , MPP

Vector Pipelines : IBM 9000, Cray X-MP, Y-MP, & C90
MISD : Multiple Instriction Stream - Single Data Stream

In MISD , multiple instructions operate on single data stream.

An MISD computing system is capable of executing different instruction

on diffrent PUs but all of them operating on the same dataset

 Multiple instruction : Each processing unit may be executing a

diffrent instruction stream

 Single data : Every processing unit can operate on a same data

element
 Machines built using the MISD model are practically not usefull in most
of the application , a few machines are built, but none of them are avalible
commercially.

Example: Systolic Arrays

 MIMD : Multiple Instruction Stream - Multiple Data
Stream
An MIMD system is capable of executing multiple instructions on multiple data
sets.

 Multiple Instruction : Every processor may be executed a diffrent

instruction streams
 Multiple Data : Every processor may be working with a diffrent data
stream

MIMD system are prallel computers capable of processing several programs

simultaneously
 Can be categorized as loosely coupled or tightly coupled depending
on sharing of data and control

Example :
most current supercomputers, networked parallel
computer “ grids” and multi-processor SMP computer

- Including some types of PC’s

IBM-370, Cary-2, Cary X-MP, C.mmp, UNIVAC - 1100/80