Synchronization free

parallelism
Muhammad Taha & Muhammad Usman
 Agenda
• Introduction
• Space partition
• Space partition constraints
• Solving Space partition constraints
• Simple code generation
• Eliminating Empty Iterations
• Eliminating tests from innermost loops

02/10/2023 PRESENTATION TITLE 2

 Introduction
for(i=0;i<=100;i++)
    {
        for(j=0;j<=100;j++)
        {
            A[i,j]=A[i+1,j-1]
        }
    }
02/10/2023 3
  A[i,j]=A[i+1,j-1]
   both accessing the same memory
location one for write and one for read.
02/10/2023 4
Parallelizable?
• It means , without having to do with
any synchronization between the
different threads/ processes that are
going to execute in parallel.
• That’s why calls sync free
parallelism
 • So as from our example the outer loop
has no data dependency
• Means it’s called embarrassingly
parallel
• So, the outer loop can be paralyzed
without any need of synchronization.
• However, the inner loop do have data
dependency,
• And this is how it looks
02/10/2023 PRESENTATION TITLE 6
 j
7
6
5
4
3
2
1
0 1 2 3 4 5 6 7 8 9

02/10/2023 7
 • Only one loop can be paralyzed not the
inner , so this way we say it has 1
degree of parallelism.
• If we have n parallelizable loops means,
we have n degree of parallelism

02/10/2023 PRESENTATION TITLE 8

Affine space partition
• To create as many virtual processors
as possible for every points in
iteration space.
• And it’s job of OS to map all these
processors on physical processors.
 Many-one-map

Iteration space Processor space

02/10/2023 PRESENTATION TITLE 10

• The range of function should be as large as possible

• For many points in an iteration space there is a unique point in processor

space

• If embarrassingly parallel, then each iteration can be assigned to separate

processor

• Else group of iteration will be given to a specific processor.

• The function from iteration space to processor space must be affine function.

• It means we will be only dealing with affine spaces whether it is iteration ,

processor or memory access.

02/10/2023 PRESENTATION TITLE 11

• If we not restrict to affine function the problem will become harder to solve

• Affine function are good enough in extracting maximum no of parallelism ,

though not always.

• Pid = Ci +c

Ci + c

Iteration space Processor space

02/10/2023 PRESENTATION TITLE 12

• Pid = Ci(coefficient matrix) +c(additive vector).

• If independent than each C,c will map to different

processor.

• Else need some constraints on C and c.

• .f(i) = Ci + c if Ri + r >= 0

• There is a trivial solution that would work to any iteration

space ,but we are interested to max the iteration space.

02/10/2023 PRESENTATION TITLE 13

 Whenever there is data dependency between two iteration of
these statements , then they should map on same processor.

Iteration space Processor space

02/10/2023 14
 Space partition constraints
 for(i=0;i<=100;i++)
    {
        for(j=0;j<=100;j++)
        {
            X[i,j] = X[i,j] + Y[i-1,j]
            Y[i,j] = Y[i,j] + X[i,j-1]
        }
    }

02/10/2023 PRESENTATION TITLE 15

 So, conflict can occur at following
statements.

X[i,j] <=> X[i,j]

X[i,j] <=> X[i,j-1]

Y[i,j] <=>Y[i,j]
Y[i,j]<=>Y[i-1,j]
02/10/2023 PRESENTATION TITLE 16
 I,j -- I’ , j’

I = I’
J = j’-1

P1 = i-j-1
P2 = i-j

When s1 =1, s2 = 0
When s1 =1 , s2 =1

02/10/2023 PRESENTATION TITLE 17

 Solving space partition

02/10/2023 PRESENTATION TITLE 18

 Simple code generation
Simple code generation is the process of automatically generating
code from a higher-level representation, such as a graphical model
or a domain-specific language. This can be used to quickly and
easily create code for a wide variety of applications.

02/10/2023 PRESENTATION TITLE 19

 Eliminating empty iterations
Eliminating empty iterations refers to the process of removing
unnecessary loop iterations from a program. This can be done by
analyzing the loop and determining if any of the iterations will not
execute any code, and then removing those iterations from the
loop.

02/10/2023 PRESENTATION TITLE 20

Thank you