Parallel computing 

is a computing where the jobs are broken into sub parts that
can be executed concurrently. Each part is further broken down to a series of
instructions. Instructions from each part execute simultaneously on different CPUs.
FLYNN’S CLASSIFICATION
This classification was proposed by Michael J Flynn in 1972.

1 . In 1966,
Michael
Flynn
proposed a
classification
for
2. computer
architectures
based on the
number of
3. instruction
streams and
data streams
4 . In 1966,
Michael
Flynn
proposed a
classification
for
5. computer
architectures
based on the
number of
6. instruction
streams and
data streams
7 . In 1966,
Michael
Flynn
proposed a
classification
for
8. computer
architectures
based on the
number of
9. instruction
streams and
data streams
1 0 . In
1966,
Michael
Flynn
proposed a
classification
for
11. computer
architectures
based on the
number of
12. instructio
n streams
 and data
streams
It is based on instruction stream and data streams in a computer system.
An instruction stream is a sequence of instruction executed by computer.
A data stream in a sequence of data which includes input data or temporary results.

They are classified into 4 types:

 SISD (Single Instruction Single Data)

 SIMD (Single Instruction Multiple Data)
 MISD (Multiple Instruction Multiple Data)
 MIMD (Multiple Instruction Multiple Data)

1) SISD (Single Instruction Single Data Stream)

A SISD computing system is a uniprocessor machine that is capable of executing a single

instruction operating on a single data stream. Most conventional computers have SISD
architecture where all the instruction and data to be processed have to be stored in
primary memory.

2) SIMD (Single Instruction Multiple Data Stream)

A SIMD system is a multiprocessor machine, capable of executing the same instruction on all
the CPUs but operating on the different data stream.
3) MISD (Multiple Instruction Single Data stream)

An MISD computing is a multiprocessor machine capable of executing different instructions

on processing elements but all of them operating on the same data set.

4) MIMD (Multiple Instruction Multiple Data Stream)

A MIMD system is a multiprocessor machine that is capable of executing multiple instructions

over multiple data streams. Each processing element has a separate instruction stream and
data stream.
Multiprocessors and Multicomputers

Multiprocessor:

A Multiprocessor is a computer system with two or more central processing units (CPUs)
share full access to a common RAM.
There are two types of multiprocessors, one is called shared memory multiprocessor and
another is distributed memory multiprocessor. In shared memory multiprocessors, all the
CPUs shares the common memory but in a distributed memory multiprocessor, every CPU
has its own private memory.

Benefits of using a Multiprocessor –

 Enhanced performance.
 Multiple applications.
 Multi-tasking inside an application.
 High throughput and responsiveness.
 Hardware sharing among CPUs.

2. Multicomputer:

A multicomputer system is a computer system with multiple processors that are

connected together to solve a problem. Each processor has its own memory and it is
accessible by that particular processor and those processors can communicate with
each other via an interconnection network.
Shared-Memory Multiprocessors
Three most common shared memory multiprocessors model is −
Uniform Memory Access (UMA)
In this model, all the processors share the physical memory uniformly. All the processors
have equal access time to all the memory words.
When all the processors have equal access to all the peripheral devices, the system is called
a symmetric multiprocessor. When only one or a few processors can access the peripheral
devices, the system is called an asymmetric multiprocessor.

Vector Supercomputers
In a vector computer, a vector processor is attached to the scalar processor as an optional
feature. The host computer first loads program and data to the main memory. Then the scalar
control unit decodes all the instructions. If the decoded instructions are scalar operations or
program operations, the scalar processor executes those operations using scalar functional
pipelines.
On the other hand, if the decoded instructions are vector operations then the instructions will
be sent to vector control unit.
SIMD Supercomputers
In SIMD computers, ‘N’ number of processors are connected to a control unit and all the
processors have their individual memory units. All the processors are connected by an
interconnection network.

Parallel Random-Access Machines

Sheperdson and Sturgis (1963) modeled the conventional Uniprocessor computers as
random-access-machines (RAM). Fortune and Wyllie (1978) developed a parallel random-
access-machine (PRAM) model for modeling an idealized parallel computer with zero
memory access overhead and synchronization.
An N-processor PRAM has a shared memory unit. This shared memory can be centralized or
distributed among the processors. These processors operate on a synchronized read-
memory, write-memory and compute cycle. So, these models specify how concurrent read
and write operations are handled.

Following are the possible memory update operations −

 Exclusive read (ER) − In this method, in each cycle only one processor is allowed to
read from any memory location.
 Exclusive write (EW) − In this method, at least one processor is allowed to write into a
memory location at a time.
 Concurrent read (CR) − It allows multiple processors to read the same information
from the same memory location in the same cycle.
 Concurrent write (CW) − It allows simultaneous write operations to the same memory
location. To avoid write conflict some policies are set up.

MODELS OF PRAM
 EREW: also called Exclusive Read Exclusive Write is a constraint that doesn’t
allow two processors to read or write from the same memory location at the same
instance.
 CREW: also called Concurrent Read Exclusive Write is a constraint that allows all
the processors to read from the same memory location but are not allowed to
write into the same memory location at the same time.
 ERCW: also called Exclusive Read Concurrent Write is a constraint that allows all
the processors to write to the same memory location but are now allowed to read
the same memory location at the same time.
 CRCW: also called Concurrent Read Concurrent Write is a constraint that allows
all the processors to read from and write to the same memory location parallelly.