Evolution of Parallel computers

Flynn’s Taxonomy

– The most universally excepted method of classifying

computer systems
– Published in the Proceedings of the IEEE in 1966
– Any computer can be placed in one of 4 broad categories
» SISD: Single instruction stream, single data stream
» SIMD: Single instruction stream, multiple data streams
» MIMD: Multiple instruction streams, multiple data streams
» MISD: Multiple instruction streams, single data stream

2
 SISD

Instructions
Processing Main memory
element (PE) (M)
Data

IS

IS DS
Control Unit PE Memory

3
SIMD
Applications:
• Image processing
• Matrix manipulations
• Sorting

4
MIMD

5
MISD

MISD called as
SYSTOLIC ARRAYS
Applications:
• Classification
• Robot vision

6
 Flynn taxonomy

– Advantages of Flynn
» Universally accepted
» Compact Notation
» Easy to classify a system (?)
– Disadvantages of Flynn
» Very coarse-grain differentiation among machine systems
» Comparison of different systems is limited
» Interconnections, I/O, memory not considered in the
scheme

7
 Contd…
• p-no of processor cycles needed for instn
decode and Exec
• m-No of memory reference needed
• K-ratio between memory cycle and processor
cycle
• Ic is the instruction count
 Classification based on memory arrangement

Shared memory
Interconnection
I/O1 network
Interconnection
network
I/On
PE1 PEn

PE1 PEn M1 Mn

Processors P1 Pn

Shared memory - multiprocessors

Message passing - multicomputers

24
 Shared-memory multiprocessors

• Uniform Memory Access (UMA)

• Non-Uniform Memory Access (NUMA)
• Cache-only Memory Architecture (COMA)

• Memory is common to all the processors.

• Processors easily communicate by means of
shared variables.

25
 The UMA Model
• Tightly-coupled systems (high degree of
resource sharing)
• Suitable for general-purpose and time-sharing
applications by multiple users.
P1 Pn

$ $

Inter connection network

Mem Mem

26
 Symmetric and asymmetric multiprocessors

• Symmetric:
- all processors have equal access to all peripheral
devices.
- all processors are identical.
• Asymmetric:
- one processor (master) executes the operating
system
- other processors may be of different types and may
be dedicated to special tasks.

27
 The NUMA Model
• The access time varies with the location of the memory word.
• Shared memory is distributed to local memories.
• All local memories form a global address space accessible by
all processors

Access time: Cache, Local memory, Remote memory

COMA - Cache-only Memory Architecture
P1 Pn

$ $
Mem Mem

Inter connection network

Distributed Memory (NUMA)

28
 Distributed memory multicomputers
• Multiple computers- nodes
• Message-passing network
• Local memories are private with its own
program and data
M M M
• No memory contention so that the number
PE PE PE
of processors is very large
• The processors are connected by
communication lines, and the precise way Interconnection
in which the lines are connected is called network
the topology of the multicomputer.
• A typical program consists of subtasks
residing in all the memories. PE PE PE
M M M

29