COMPUTER SYSTEM ARCHITECTURE - CS 405

Module -1 Part - 2

Conditions of Parallelism
 Conditions of Parallelism: Data & Name Dependence
S1
.. Assume task S2 follows task S1 in sequential program order
.. 1 True Data or Flow Dependence (Read After Write (RAW) Dependence):
S2 Task S2 is data dependent on task S1 if an execution path exists
Program
from S1 to S2 and if at least one output variable of S1 feeds in as
Order an input operand of S2
Represented by S1 S2 in dependency graphs

2 Antidependence: Task S2 is antidependent on S1, if S2 follows S1

in program order and if the output of S2 overlaps the input of S1
Name
Dependencies
Represented by S1 S2 in dependency graphs

3 Output dependence: Two tasks S1, S2 are output dependent

if they produce the same output variable
Represented by S1  S2 in dependency graphs
 Bernstein’s Conditions: An Example
EG: Determine the parallelism embedded in the following statements and draw the
dependency graphs. Also analyse the statements against Bernstein’s Conditions.
P1: C = D x E
P2: M=G+C
P3: A=B+C
P4: C=L+M
P5: F = G  E
Using Bernstein’s Conditions after checking statement pairs P1 || P2 , P1|| P3 etc …
dependency graph is drawn as below
Bernstein’s Conditions: An Example
 Hardware and Software Parallelism

• Hardware parallelism: e.g Number of processors

– Defined by machine architecture, hardware multiplicity (number of processors available) and
connectivity.
– Displays Resource Utilization Patterns of simultaneously executable operations
– Indicate Peak performance of processor resources.
– Often a function of cost/performance tradeoffs.
– Characterized in a single processor by the number of instructions k issued in a single cycle (k-
issue processor).
– A multiprocessor system with n k-issue processor can handle a maximum limit of nk parallel
instructions (at ILP level) or n parallel threads at thread-level parallelism (TLP) level.
– Representative systems: Intel i960CA (3-issue proc- arith, mem access, branch), IBM RISC
System 6000 (4-issue proc – floating point, arith, mem access, branch)
• Software parallelism: e.g Degree of Parallelism (DOP)

– Defined by the control and data dependence of programs.

– Revealed in program profiling or program dependency (data flow) graph.
– A function of algorithm, parallel task grain size, programming style and compiler
optimization.
– Program Flow Graphs - Patterns of simultaneously executable operations
– Parallelism in a program varies during execution period
– Types -Control Parallelism and Data Parallelism