CSC429

Computer Organization and Architecture

Chapter 6
ADVANCED ARCHITECTURES
Parallel Organization

Edited by
Salehah Hamzah/Siti Fatimah Mohd Rum
Faculty of Computer Science and Mathematics UiTM Melaka, Kampus Jasin
INTRODUCTION TO PARALLEL SYSTEM
▪ Flynn's taxonomy propose by Michael J. Flynn (1966)
a classification of parallel computer architectures based on number of
instructions and number of data streams (single and multiple)
▪ Parallel system is simultaneously uses of more than one CPU to execute
a program
▪ A parallel computer is a supercomputer with hundreds or thousands of
processors or may be a network of workstations, specialized
application that require large amount of mathematical calculations
MULTIPLE
PROCESSO
R
ORGANIZA
TION
▪ To avoid the overhead of processing loops and fasten the computation, some kind of parallelism must
be introduced. Vector processing operates on the entire array in just one operation i.e. it operates on
elements of the array in parallel. But, vector processing is possible only if the operations performed in
parallel are independent.
▪ Look at the figure below, and compare the vector processing with the general computer processing,
you will notice the difference. Below, instructions in both the blocks are set to add two arrays and
store the result in the third array. Vector processing adds both the array in parallel by avoiding the use
of the loop
APPLICATIONS

▪ numeric weather forecasting,

▪ Natural disaster forecasting(e.g: earthquake, hurricane,
tsunami)
▪ Testing nuclear weapon
▪ medical image processing( human brains)
▪ remote sensing application (sense earth resources at
forestry,agricultural)
▪ energy resource explorations (e.g : oil and gas)
▪ Geography Information System (e.g : image data, map data)
 SISD AND SIMD ARCHITECTURE

A Von Neumann model

SIMD operate on specialized machines. They are designed to solve numerical

problems comprising substantial numbers of vector and matrix elements
SIMD ARCHITECTURE
 A sequence of data is transmitted to a set of processors,
MISD ARCHITECTURE each of which executes a different instruction
sequence. This structure is not commercially
implemented.
MIMD ARCHITECTURE
 CATEGORIES OF MIMD
▪ Parallel architectures can be classified into two major
categories in terms of memory arrangement.
▪ shared memory
▪ distributed memory
▪ Shared memory multiprocessors can be of two types
1. uniform memory access (UMA) architecture
2. non-uniform memory access (NUMA) architecture
TIGHTLY COUPLED VS LOOSELY COUPLED
SYMMETRIC MULTIPROCESSOR (SMP) OR UMA

▪ Uniform Memory Access (UMA) also called as Symmetric

multiprocessor (SMP)
▪ There are two or more similar processors.
▪ share main memory and I/O facilities and connected by
a shared BUS
▪ All processors can perform the same functions under control of
an integrated operating system
NUMA ARCHITECTURE

▪ The interconnection of SMP with another SMP through

Interconnection network(s) are known as NUMA (non
uniform memory access).
▪ UMA is best for not more then 8 processors due to
scalability issue.
▪ NUMA makes more than 8 processors, with every unit of
processors have their own local physical memory which is
easy to access but logically there is one address shared
space.
COMPARISON BETWEEN SMP AND DISTRIBUTED
MEMORY
▪ Shared architectures are easy to program as they provide a single
address space as found in uniprocessor environment.
▪ In shared architecture, any failing processor or memory can cause the
whole system to fail due to the integration.
▪ Shared architecture difficult to add new processors in particular
(poor scalability)
▪ Distributed architecture give excellent scalability
▪ Distributed memory machines pose difficulty due to multiple address
spaces
 CLUSTER
▪ a group of interconnected, whole computers working
together as a unified computing resource that can create
the illusion of being one machine. Cluster can have tens,
hundreds, or even thousands of machines which each
one is a multiprocessor.
▪ E.g: A Hadoop cluster - a computational computer cluster for storing and analysing big data
(structured, semi-structured and unstructured) in a distributed environment. Face book and
Yahoo use Hadoop cluster architecture.
▪ E.g Facebook has a Hadoop warehouse with two level network topology having 4800 cores,
5.5 PB storing up to 12TB per node.
▪ Other cluster : Google cluster
BENEFITS OF CLUSTER ORGANIZATION

1. High processing speed

2. Offer scalability -easily expanded by adding additional
nodes to the network)
3. Provide high availability of resources - can act as
backup system in the event of failure of system
4. Processing power is cost effective compare to
mainframe
5. Drawback : High cost to implement and maintain
CLUSTER ORGANIZATION

Middleware is a software layer for integrating software applications that run in a

heterogeneous environment
CLUSTER ARCHITECTURE IN DISTRIBUTED COMPUTING ENVIRONMENT

http://saphanatutorial.com/hadoop-cluster-architecture-and-core-components/
 HADOOP CLUSTER ARCHITECTURE

http://saphanatutorial.com/hadoop-cluster-architecture-and-core-components/