Part01 PDF

Technische Universität München
Parallel Programming
and High-Performance Computing
Part 1: Introduction
Dr. Ralf-Peter Mundani

CeSIM / IGSSE
1 Introduction
General Remarks
• materials: http://www5.in.tum.de/lehre/vorlesungen/parhpp/SS08/
• Ralf-Peter Mundani
– email mundani@tum.de, phone 289–25057, room 3181 (city centre)
– consultation-hour: Tuesday, 4:00—6:00 pm (room 02.05.058)
• Ioan Lucian Muntean
– email muntean@in.tum.de, phone 289–18692, room 02.05.059
• lecture (2 SWS)
– weekly
– Tuesday, start at 12:15 pm, room 02.07.023
• exercises (1 SWS)
– fortnightly
– Wednesday, start at 4:45 pm, room 02.07.023
Dr. Ralf-Peter Mundani - Parallel Programming and High-Performance Computing - Summer Term 2008 1−2
1 Introduction
General Remarks
• content
– part 1: introduction
– part 2: high-performance networks
– part 3: foundations
– part 4: programming memory-coupled systems
– part 5: programming message-coupled systems
– part 6: dynamic load balancing
– part 7: examples of parallel algorithms
1 Introduction
Overview
• motivation
• classification of parallel computers
• levels of parallelism
• quantitative performance evaluation
I think there is a world market

for maybe five computers.
—Thomas Watson,
chairman IBM, 1943
1 Introduction
Motivation
• numerical simulation: from phenomena to predictions
physical phenomenon
technical process
1. modelling
determination of parameters, expression of relations
2. numerical treatment
model discretisation, algorithm development
3. implementation
software development, parallelisation
discipline
4. visualisation
mathematics illustration of abstract simulation results
computer science 5. validation

comparison of results with reality
application 6. embedding
insertion into working process
1 Introduction
Motivation
• why parallel programming and HPC?
– complex problems (especially the so called “grand challenges”) demand
for more computing power
• climate or geophysics simulation (tsunami, e. g.)
• structure or flow simulation (crash test, e. g.)
• development systems (CAD, e. g.)
• large data analysis (Large Hadron Collider at CERN, e. g.)
• military applications (crypto analysis, e. g.)
• …
– performance increase due to
• faster hardware, more memory (“work harder”)
• more efficient algorithms, optimisation (“work smarter”)
• parallel computing (“get some help”)
1 Introduction
Motivation
• objectives (in case all resources would be available N-times)
– throughput: compute N problems simultaneously
• running N instances of a sequential program with different data sets
(“embarrassing parallelism”); SETI@home, e. g.
• drawback: limited resources of single nodes
– response time: compute one problem at a fraction (1/N) of time
• running one instance (i. e. N processes) of a parallel program for
jointly solving a problem; finding prime numbers, e. g.
• drawback: writing a parallel program; communication
– problem size: compute one problem with N-times larger data
• running one instance (i. e. N processes) of a parallel program, using
the sum of all local memories for computing larger problem sizes;
iterative solution of SLE, e. g.
• drawback: writing a parallel program; communication
1 Introduction
Overview
• motivation
1 Introduction
Classification of Parallel Computers
• definition: “A collection of processing elements that communicate and
cooperate to solve large problems” (ALMASE and GOTTLIEB, 1989)
• possible appearances of such processing elements
– specialised units (steps of a vector pipeline, e. g.)
– parallel features in modern monoprocessors (superscalar architectures,
instruction pipelining, VLIW, multithreading, multicore, …)
– several uniform arithmetical units (processing elements of array
computers, e. g.)
– processors of a multiprocessor computer (i. e. the actual parallel
computers)
– complete stand-alone computers connected via LAN (work station or PC
clusters, so called virtual parallel computers)
– parallel computers or clusters connected via WAN (so called
metacomputers)
1 Introduction
• reminder: dual core, quad core, manycore, and multicore
– observation: increasing frequency (and thus core voltage) over past years
– problem: thermal power dissipation increases linearly in frequency and
with the square of the core voltage
1 Introduction
• reminder: dual core, quad core, manycore, and multicore (cont’d)
– 25% reduction in frequency (and thus core voltage) leads to 50%
reduction in dissipation
dissipation
performance
Î
normal CPU reduced CPU
1 Introduction
• reminder: dual core, quad core, manycore, and multicore (cont’d)
– idea: installation of two cores per die with same dissipation as single
core system
dissipation
performance
Î
single core dual core
1 Introduction
• commercial parallel computers
– manufacturers: starting from 1983, big players and small start-ups (see
tabular; out of business: no longer in the parallel business)
– names have been coming and going rapidly
– in addition: several manufacturers of vector computers and non-
standard architectures
company country year status in 2003

Sequent U.S. 1984 acquired by IBM
Intel U.S. 1984 out of business
Meiko U.K. 1985 bankrupt
nCUBE U.S. 1985 out of business
Parsytec Germany 1985 out of business
Alliant U.S. 1985 bankrupt
1 Introduction
• commercial parallel computers (cont’d)
company country year status in 2003

Encore U.S. 1986 out of business
Floating Point Systems U.S. 1986 acquired by SUN
Myrias Canada 1987 out of business
Ametek U.S. 1987 out of business
Silicon Graphics U.S. 1988 active
C-DAC India 1991 active
Kendall Square Research U.S. 1992 bankrupt
IBM U.S. 1993 active
NEC Japan 1993 active
SUN Microsystems U.S. 1993 active
Cray Research U.S. 1993 active
1 Introduction
• arrival of clusters
– in the late eighties, PCs became a commodity market with rapidly
increasing performance, mass production, and decreasing prices
– growing attractiveness for parallel computers
– 1994: Beowulf, the first parallel computer built completely out of
commodity hardware
• NASA Goddard Space Flight Centre
• 16 Intel DX4 processors
• multiple 10 Mbit Ethernet links
• Linux with GNU compilers
• MPI library
– 1996: Beowulf cluster performing more than 1 GFlops
– 1997: a 140-node cluster performing more than 10 GFlops
1 Introduction
• arrival of clusters (cont’d)
– 2005: InfiniBand cluster at TUM
• 36 Opteron nodes (quad boards)
• 4 Itanium nodes (quad boards)
• 4 Xeon nodes (dual boards) for interactive tasks
• InfiniBand 4× Switch, 96 ports
• Linux (SuSE and Redhat)
1 Introduction
• supercomputers
– supercomputing or high-performance scientific computing as the most
important application of the big number crunchers
– national initiatives due to huge budget requirements
• Accelerated Strategic Computing Initiative (ASCI) in the U.S.
– in the sequel of the nuclear testing moratorium in 1992/93
– decision: develop, build, and install a series of five
supercomputers of up to $100 million each in the U.S.
– start: ASCI Red (1997, Intel-based, Sandia National Laboratory,
the world’s first TFlops computer)
– then: ASCI Blue Pacific (1998, LLNL), ASCI Blue Mountain, ASCI
White, …
• meanwhile new high-end computing memorandum (2004)
1 Introduction
• supercomputers (cont’d)
– federal “Bundeshöchstleistungsrechner” initiative in Germany
• decision in the mid-nineties
• three federal supercomputing centres in Germany (Munich,
Stuttgart, and Jülich)
• one new installation every second year (i. e. a six year upgrade cycle
for each centre)
• the newest one to be among the top 10 of the world
– overview and state of the art: Top500 list (updated every six month), see
http://www.top500.org
1 Introduction
• MOORE’s law
– observation of Intel co-founder Gordon E. MOORE,
describes important trend in history of computer
hardware (1965)
number of transistors that can be

placed on an integrated circuit is
increasing exponentially, doubling
approximately every two years
1 Introduction
• some numbers: Top500
1 Introduction
• some numbers: Top500 (cont’d)
1 Introduction
cluster: #nodes > #processors/node

constellation: #nodes < #processors/node
1 Introduction
1 Introduction
1 Introduction
1 Introduction
1 Introduction
1 Introduction
1 Introduction
1 Introduction
• The Earth Simulator – world’s #1 from 2002—04
– installed in 2002 in Yokohama, Japan
– ES-building (approx. 50m × 65m × 17m)
– based on NEC SX-6 architecture
– developed by three governmental agencies
– highly parallel vector supercomputer
– consists of 640 nodes (plus 2 control & 128 data switching)
• 8 vector processors (8 GFlops each)
• 16 GB shared memory
Î 5120 processors (40.96 TFlops peak performance) and 10 TB
memory; 35.86 TFlops sustained performance (Linpack)
– nodes connected by 640×640 single stage crossbar (83,200 cables with
a total extension of 2400km; 8 TBps total bandwidth)
– further 700 TB disc space and 1.6 PB mass storage
1 Introduction
• BlueGene/L – world’s #1 since 2004
– installed in 2005 at LLNL, CA, USA
(beta-system in 2004 at IBM)
– cooperation of DoE, LLNL, and IBM
– massive parallel supercomputer
– consists of 65,536 nodes (plus 12 front-end and 1204 I/O nodes)
• 2 PowerPC 440d processors (2.8 GFlops each)
• 512 MB memory
Î 131,072 processors (367 TFlops peak performance) and
33.5 TB memory; 280.6 TFlops sustained performance (Linpack)
– nodes configured as 3D torus (32 × 32 × 64); global reduction tree for
fast operations (global max / sum) in a few microseconds
– 1024 Gbps link to global parallel file system
– further 806 TB disc space; operating system SuSE SLES 9
1 Introduction
• HLRB II (world’s #6 for 04/2006)
– installed in 2006 at LRZ, Garching
– installation costs 38 M€
– monthly costs approx. 400,000 €
– upgrade in 2007 (finished)
– one of Germany’s 3 supercomputers
– SGI Altix 4700
– consists of 19 nodes (SGI NUMA link 2D torus)
• 256 blades (ccNUMA link with partition fat tree)
– Intel Itanium2 Montecito Dual Core (12.8 GFlops)
– 4 GB memory per core
Î 9728 processor cores (62.3 TFlops peak performance) and 39 TB
memory; 56.5 TFlops sustained performance (Linpack)
– footprint 24m × 12m; total weight 103 metric tons
1 Introduction
• standard classification according to FLYNN
– global data and instruction streams as criterion
• instruction stream: sequence of commands to be executed
• data stream: sequence of data subject to instruction streams
– two-dimensional subdivision according to
• amount of instructions per time a computer can execute
• amount of data elements per time a computer can process
– hence, FLYNN distinguishes four classes of architectures
• SISD: single instruction, single data
• SIMD: single instruction, multiple data
• MISD: multiple instruction, single data
• MIMD: multiple instruction, multiple data
– drawback: very different computers may belong to the same class
1 Introduction
• standard classification according to FLYNN (cont’d)
– SISD
• one processing unit that has access to one data memory and to one
program memory
• classical monoprocessor following VON NEUMANN’s principle
data memory processor program memory
1 Introduction
– SIMD
• several processing units, each with separate access to a (shared or
distributed) data memory; one program memory
• synchronous execution of instructions
• example: array computer, vector computer
• advantages: easy programming model due to control flow with a
strict synchronous-parallel execution of all instructions
• drawbacks: specialised hardware necessary, easily becomes out-
dated due to recent developments at commodity market
data memory processor
program memory
data memory processor
1 Introduction
– MISD
• several processing units that have access to one data memory;
several program memories
• not very popular class (mainly for special applications such as Digital
Signal Processing)
• operating on a single stream of data, forwarding results from one
processing unit to the next
• example: systolic array (network of primitive processing elements
that “pump” data)
processor program memory
data memory
processor program memory
1 Introduction
– MIMD
• several processing units, each with separate access to a (shared or
distributed) data memory; several program memories
• classification according to (physical) memory organisation
– shared memory Î shared (global) address space
– distributed memory Î distributed (local) address space
• example: multiprocessor systems, networks of computers
1 Introduction
• processor coupling
– cooperation of processors / computers as well as their shared use of
various resources require communication and synchronisation
– the following types of processor coupling can be distinguished
• memory-coupled multiprocessor systems (MemMS)
• message-coupled multiprocessor systems (MesMS)
global memory distributed memory
shared
MemMS, SMP Mem-MesMS (hybrid)
address space
distributed
∅ MesMS
address space
1 Introduction
• processor coupling (cont’d)
– central issues
• scalability: costs for adding new nodes / processors
• programming model: costs for writing parallel programs
• portability: costs for portation (migration), i. e. transfer from one
system to another while preserving executability and flexibility
• load distribution: costs for obtaining a uniform load distribution
among all nodes / processors
– MemMS are advantageous concerning scalability, MesMS are typically
better concerning the rest
– hence, combination of MemMS and MesMS for exploiting all
advantages Î distributed / virtual shared memory (DSM / VSM)
– physical distributed memory with global shared address space
1 Introduction
– uniform memory access (UMA)
• each processor P has direct access via the network to each memory
module M with same access times to all data
• standard programming model can be used (i. e. no explicit send /
receive of messages necessary)
• communication and synchronisation via shared variables
(inconsistencies (write conflicts, e. g.) have to prevented in general
by the programmer)
M M … M
network
P P … P
1 Introduction
– symmetric multiprocessor (SMP)
• only a small amount of processors, in most cases a central bus, one
address space (UMA), but bad scalability
• cache-coherence implemented in hardware (i. e. a read always
provides a variable’s value from its last write)
• example: double or quad boards, SGI Challenge
C: cache
C C C
…
P P P
1 Introduction
– non-uniform memory access (NUMA)
• memory modules physically distributed among processors
• shared address space, but access times depend on location of data
(i. e. local addresses faster than remote addresses)
• differences in access times are visible in the program
• example: DSM / VSM, Cray T3E
network
M M
P … P
1 Introduction
– cache-coherent non-uniform memory access (ccNUMA)
• caches for local and remote addresses; cache-coherence
implemented in hardware for entire address space
• problem with scalability due to frequent cache actualisations
• example: SGI Origin 2000
network
M M
C … C
P P
1 Introduction
– cache-only memory access (COMA)
• each processor has only cache-memory
• entirety of all cache-memories = global shared memory
• cache-coherence implemented in hardware
• example: Kendall Square Research KSR-1
network
C C C
…
P P P
1 Introduction
– no remote memory access (NORMA)
• each processor has direct access to its local memory only
• access to remote memory only via explicit message exchange (due
to distributed address space) possible
• synchronisation implicitly via the exchange of messages
• performance improvement between memory and I/O due to parallel
data transfer (Direct Memory Access, e. g.) possible
• example: IBM SP2, ASCI Red / Blue / White
network
P P P
…
M M M
1 Introduction
Overview
• motivation
1 Introduction
Levels of Parallelism
• the suitability of a parallel architecture for a given parallel program strongly
depends on the granularity of parallelism
• some remarks on granularity
– quantitative meaning: ratio of computational effort and communication /
synchronisation effort (≈ amount of instructions between two necessary
communication / synchronisation steps)
– qualitative meaning: level on which work is done in parallel
parallelism
program level
fine-grain
process level
coarse-grain
parallelism
block level
instruction level
sub-instruction level
1 Introduction
• program level
– parallel processing of different programs
– independent units without any shared data
– no or only small amount of communication / synchronisation
– organised by the OS
• process level
– a program is subdivided into processes to be executed in parallel
– each process consists of a larger amount of sequential instructions and
has a private address space
– synchronisation necessary (in case all processes in one program)
– communication in most cases necessary (data exchange, e. g.)
– support by OS via routines for process management, process
communication, and process synchronisation
– term of process often referred to as heavy-weight process
1 Introduction
• block level
– blocks of instructions are executed in parallel
– each block consists of a smaller amount of instructions and shares the
address space with other blocks
– communication via shared variables; synchronisation mechanisms
– term of block often referred to as light-weight-process (thread)
• instruction level
– parallel execution of machine instructions
– optimising compilers can increase this potential by modifying the order
of commands (better exploitation of superscalar architecture and
pipelining mechanisms)
• sub-instruction level
– instructions are further subdivided in units to be executed in parallel or
via overlapping (vector operations, e. g.)
1 Introduction
Overview
• motivation
1 Introduction
Quantitative Performance Evaluation
• execution time
– time T of a parallel program between start of the execution on one
processor and end of all computations on the last processor
– during execution all processors are in one of the following states
• compute
– computation time TCOMP
– time spent for computations
• communicate
– communication time TCOMM
– time spent for send and receive operations
• idle
– idle time TIDLE
– time spent for waiting (sending / receiving messages)
– hence T = TCOMP + TCOMM + TIDLE
1 Introduction
• parallel profile
– measures the amount of parallelism of a parallel program
– graphical representation
• x-axis shows time, y-axis shows amount of parallel activities
• identification of computation, communication, and idle periods
– example
proc. A
proc. B
proc. C
compute
3
processes
amount of
communicate
2
idle
1
0 time
1 Introduction
• parallel profile (cont’d)
– degree of parallelism
• P(t) indicates the amount of processes (of one application) that can
be executed in parallel at any point in time (i. e.
y-values of the previous example for any time t)
– average parallelism (often referred to as parallel index)
• A(p) indicates the average amount of processes that can be
executed in parallel, hence
∑i=1i ⋅ ti ,
t p
2
1
A(p) = ⋅ ∫ P(t)dt or A(p) =
t 2 − t1 t1 ∑i=1 ti
p
where p is the amount of processes and ti is the time when exactly i

processes are busy
1 Introduction
• parallel profile (cont’d)
– previous example: A(p) = (1⋅18 + 2⋅4 + 3⋅13) / 35 = 65/35 = 1.86
P(t)
3
2
1
time
5 10 15 20 25 30 35 40 45
– for A(p) exist several theoretical (typically quite pessimistic) estimates,
often used as arguments against parallel systems
– example: estimate of MINSKY (1971)
• problem: amount of used processors is halved in every step
• parallel summation of 2p numbers on p processors, e. g.
• result?
1 Introduction
• comparison multiprocessor / monoprocessor
– correlation of multi- and monoprocessor systems’ performance
– important: program that can be executed on both systems
– definitions
• P(1): amount of unit operations of a program on the monoprocessor
system
• P(p): amount of unit operations of a program on the multiprocessor
systems with p processors
• T(1): execution time of a program on the monoprocessor system
(measured in steps or clock cycles)
• T(p): execution time of a program on the multiprocessor system
(measured in steps or clock cycles) with p processors
1 Introduction
• comparison multiprocessor / monoprocessor (cont’d)
– simplifying preconditions
• T(1) = P(1)
– one operation to be executed in one step on the monoprocessor
system
• T(p) ≤ P(p)
– more than one operation to be executed in one step (for p ≥ 2)
on the multiprocessor system with p processors
1 Introduction
– speed-up
• S(p) indicates the improvement in processing speed
T(1)
S(p) =
T(p)
• in general, 1 ≤ S(p) ≤ p
– efficiency
• E(p) indicates the relative improvement in processing speed
S(p)
E(p) =
p
• improvement is normalised by the amount of processors p
• in general, 1/p ≤ E(p) ≤ 1
1 Introduction
– speed-up and efficiency can be seen in two different ways
• algorithm-independent
– best known sequential algorithm for the monoprocessor system
is compared to the respective parallel algorithm for the
multiprocessor system
Î absolute speed-up
Î absolute efficiency
• algorithm-dependent
– parallel algorithm is treated as sequential one to measure the
execution time on the monoprocessor system; “unfair” due to
communication and synchronisation overhead
Î relative speed-up
Î relative efficiency
1 Introduction
– overhead
• O(p) indicates the necessary overhead of a multiprocessor system
for organisation, communication, and synchronisation
P(p)
O(p) =
P(1)
• in general, 1 ≤ O(p)
– parallel index
• I(p) indicates the amount of operations executed on average per time
unit
P(p)
I(p) =
T(p)
• I(p) ≈ relative speed-up (taking into account the overhead)
1 Introduction
– utilisation
• U(p) indicates the amount of operations each processor executes on
average per time unit
I(p)
U(p) =
p
• conforms to the normalised parallel index
– conclusions
• all defined expressions have a value of 1 for p = 1
• the parallel index is an upper bound for the speed-up
1 ≤ S(p) ≤ I(p) ≤ p
• the workload is an upper bound for the efficiency
1/p ≤ E(p) ≤ U(p) ≤ 1
1 Introduction
– example (1)
• a monoprocessor systems needs 6000 steps for the execution of
6000 operations to compute some result
• a multiprocessor system with five processors needs 6750 operations
for the computation of the same result, but it needs only 1500 steps
for the execution
• thus P(1) = T(1) = 6000, P(5) = 6750, and T(5) = 1500
• speed-up and efficiency can be computed as
S(5) = 6000/1500 = 4 and E(5) = 4/5 = 0.8
Î there is an acceleration of factor 4 compared to the

monoprocessor system, i. e. on average an improvement of 80% for
each processor of the multiprocessor system
1 Introduction
– example (2)
• parallel index and utilisation can be computed as
I(5) = 6750/1500 = 4.5 and U(5) = 4.5/5 = 0.9
Î on average 4.5 processors are simultaneously busy, i. e. each

processor is working only for 90% of the execution time
• overhead can be computed as
O(5) = 6750/6000 = 1.125
Î there is an overhead of 12.5% on the multiprocessor system

compared to the monoprocessor system
1 Introduction
• scalability
– objective: adding further processing elements to the system shall reduce
the execution time without any program modifications
– i. e. a linear performance increase with an efficiency close to 1
– important for the scalability is a sufficient problem size
• one porter may carry one suitcase in a minute
• 60 porters won’t do it in a second
• but 60 porters may carry 60 suitcases in a minute
– in case of a fixed problem size and an increasing amount of processors
saturation will occur for a certain value of p, hence scalability is limited
– when scaling the amount of processors together with the problem size
(so called scaled problem analysis) this effect will not appear for good
scalable hard- and software systems
1 Introduction
• AMDAHL’s law
– the probably most important and most famous estimate for the speed-
up (even if quite pessimistic)
– underlying model
• each program consists of a sequential part s, 0 ≤ s ≤ 1, that can only
be executed in a sequential way; synchronisation, data I/O, e. g
• furthermore, each program consists of a parallelisable part 1−s that
can be executed in parallel by several processes; finding the
maximum value within a set of numbers, e. g.
– hence, the execution time for the parallel program executed on p
processors can be written as
1− s
T(p) = s ⋅ T(1) + ⋅ T(1)
p
1 Introduction
• AMDAHL’s law (cont’d)
– the speed-up can thus be computed as
T(1) T(1) 1
S(p) = = =
T(p) 1− s 1− s
s ⋅ T(1) + ⋅ T(1) s+
p p
– when increasing p → ∞ we finally get AMDAHL’s law
1 1
lim S(p) = lim =
p →∞ p →∞ 1− s s
s+
p
Î speed-up is bounded: S(p) ≤ 1/s
– the sequential part can have a dramatic impact on the speed-up
– therefore central effort of all (parallel) algorithms: keep s small
– many parallel programs have a small sequential part (s < 0.1)
1 Introduction
• AMDAHL’s law (cont’d)
– example
• s = 0.1 and, thus, S(p) ≤ 10
• independent from p the speed-up is bounded by this limit
• where’s the error?
S(p)
10
s = 0.1
5 10 15 20 25 p
1 Introduction
• GUSTAFSON’s law
– addresses the shortcomings of AMDAHL’s law as it states that any
sufficient large problem can be efficiently parallelised
– instead of a fixed problem size it supposes a fixed time concept
– underlying model
• execution time on the parallel machine is normalised to 1
• this contains a non-parallelisable part σ, 0 ≤ σ ≤ 1
– hence, the execution time for the sequential program on the
monoprocessor can be written as
T(1) = σ + p⋅(1−σ)
– the speed-up can thus be computed as
S(p) = σ + p⋅(1−σ) = p + σ⋅(1−p)
1 Introduction
• GUSTAFSON’s law (cont’d)
– difference to AMDAHL
• sequential part s(p) is not constant, but gets smaller with increasing p
σ
s(p) = , s(p) ∈ ]0, 1[
σ + p ⋅ (1− σ )
• often more realistic, because more processors are used for a larger
problem size, and here parallelisable parts typically increase (more
computations, less declarations, …)
• speed-up is not bounded for increasing p
1 Introduction
• GUSTAFSON’s law (cont’d)
– some more thoughts about speed-up
• theory tells: a superlinear speed-up does not exist
– each parallel algorithm can be simulated on a monoprocessor
system by emulating in a loop always the next step of a
processor from the multiprocessor system
• but superlinear speed-up can be observed
– when improving an inferior sequential algorithm
– when a parallel program (that does not fit into the main memory
of the monoprocessor system) completely runs in cache and
main memory of the nodes from the multiprocessor system
1 Introduction
• communication—computation-ratio (CCR)
– important quantity measuring the success of a parallelisation
• gives the relation of pure communication time and pure computing
time
• a small CCR is favourable
• typically: CCR decreases with increasing problem size
– example
• N×N matrix distributed among p processors (N/p rows each)
• iterative method: in each step, each matrix element is replaced by
the average of its eight neighbour values
• hence, the two neighbouring rows are always necessary
• computation time: 8N⋅N/p
• communication time: 2N
• CCR: p/4N – what does this mean?

Part01 PDF

Uploaded by

Document Information

Original Title

Copyright

Available Formats

Share this document

Share or Embed Document

Sharing Options

Did you find this document useful?

Is this content inappropriate?

Copyright:

Available Formats

Part01 PDF

Uploaded by

Copyright:

Available Formats

Technische Universität München

Dr. Ralf-Peter Mundani

I think there is a world market

computer science 5. validation

normal CPU reduced CPU

single core dual core

company country year status in 2003

company country year status in 2003

number of transistors that can be

cluster: #nodes > #processors/node

data memory processor program memory

data memory processor

processor program memory

data memory processor program memory

data memory processor program memory

global memory distributed memory

where p is the amount of processes and ti is the time when exactly i

S(5) = 6000/1500 = 4 and E(5) = 4/5 = 0.8

Î there is an acceleration of factor 4 compared to the

I(5) = 6750/1500 = 4.5 and U(5) = 4.5/5 = 0.9

Î on average 4.5 processors are simultaneously busy, i. e. each

O(5) = 6750/6000 = 1.125

Î there is an overhead of 12.5% on the multiprocessor system

– the speed-up can thus be computed as

S(p) = σ + p⋅(1−σ) = p + σ⋅(1−p)

You might also like