Proceedings of the Fifth International Conference on Computing Methodologies and Communication (ICCMC 2021)
IEEE Xplore Part Number: CFP21K25-ART
Parallel Programming Models and Paradigms:
OpenMP Analysis
2021 5th International Conference on Computing Methodologies and Communication (ICCMC) | 978-1-6654-0360-3/20/$31.00 ©2021 IEEE | DOI: 10.1109/ICCMC51019.2021.9418401
Arwa Alrawais
College of Computer Engineering and Sciences
Prince Sattam Bin Abdulaziz University
Al-Kharj 11942, Saudi Arabia.
Email: a.alrawais@psau.edu.sa
Abstract—The increase demand for processing power has
grown over the years, this demand lend to the parallel approach
which means linking a bunch of computers together to jointly
increase both the speed and efficiency. The parallelism approach
plays a significant role in the new generation’s applications by
moving the technologies from expensive and specialized parallel
supercomputers to linking a set of computers. Throughout the
years, the parallel approach lends to parallel programming
models which exists above hardware and memory architectures. It
is a collection of software technologies that present parallel algo-
rithms and resembling applications with underlying system. This
paper describes the essential concept of parallel programming
and a brief overview of different areas of parallel programming
models and paradigms. Furthermore, it implements and evaluates
OpenMP parallel programming and illustrates its effectiveness.
Keywords—Parallel programming models; Parallel program-
ming paradigms; Parallel algorithm; Parallel applications
I. I NTRODUCTION
In 1980’s, the computer at that time was thought, the best
computer in its performance by making the computer faster
and adding more efficient processors. Throughout the years,
the shifting to parallel processing that essentially attempt to
solve a computation problem made the computer performance
different than it was. In the earliest of 1990’s, the endeavor
in transition from expensive and massively parallel processors
that contained some supercomputers toward the networks of
computers was growth sharply. As a consequence, the rising
lend to the availability of the high performance computers
such as PC’s, networks and workstation at the market. The
making of the cluster or network of computers, additionally,
are being attractive for many people because the cost effective
of the parallel processing. At the same time of developing
the hardware demands, there was a progress in providing
programming resources to be attuned with diversity of com-
puting environments. The programming development directed
to numerous of parallel programming models which permits
the expression of parallel to be executed [1].
The parallel programming models, generally, can be as-
sessed by the range of the problems that can be exhibited and
how these models can be executed in several architectures.
In choosing a parallel programming model, considering how
much parallelism can be exploited is an important aspect.
This paper will propose the parallel programming models
from various aspects. Furthermore, it will explain the different
phases of designing parallel algorithm as well as different
Authorized licensed use limited to: Nitte Meenakshi Institute of Technology. Downloaded on October 27,2022 at 13:21:27 UTC from IEEE Xplore. Restrictions apply.
parallel programming paradigms. Finally, some of the parallel
programming application will be represented, such as OpenMP.
There is a miss conceptual among most of the programmers
that writing a parallel programming is hard, because of that
most of the programmer prefer writing a sequential program
than writing a parallel program. “The parallel programming
model exists as abstraction above hardware and memory
architecture [2]”. It attempts to express the parallel program,
and exploit the parallelism to solve a problem. The models are
different from each other, each can execute a different range of
problems, and each can be run in different architecture. There
are two main approaches for parallel programming:
• Implicit parallelism: when the programmer does not
specify the parallelism, therefore cannot control com-
putation scheduling or the data placement.
• Explicit parallelism: The parallelism is explicitly spec-
ified in the program code by the programmer using
some tools, such as library cells. This approach per-
mits the user to assess how much parallelism can
be exploited. The efficiency that is obtained by the
explicitly parallelism is better than that is obtained by
the implicitly parallelism [1].
The rest of this paper is organized as follows. In section II,
the paper summarize the related works to parallel models and
paradigms. Section III introduces several programming models
and tools. An implementation and evaluation of OpenMP is
conducted in Section VII. At last, a conclusion is drawn VIII.
II. R ELATED W ORK
Over past few decades, several parallel programming mod-
els and paradigms have been researched. In 1993, [3] Giloi,
W.K. formed the parallel programming domain in memory
sharing model or message passing paradigms. He indicated
the advantage and disadvantages for each parallel program-
ming models as well as its appropriate architecture. At that
time, the attention of most of the researchers and developers
were toward more to the parallel programming models, as a
result there were many conferences and workshop held. The
proceeding third working conference on massively parallel
programming models, and the proceedings third international
workshop on high-level parallel programming models and sup-
portive environments HIP’98 were held for example in 1997
and 1998. Consequently, the mature improvement throughout
the years on the parallel programming research area lend
1022
to the development and the appearance of numerous parallel
programming models and paradigms.
Over the years, there were many new parallel programming
models. In 2009 [4], MSI (Multi-thread Schedule Interface), a
new parallel programming model has been proposed to utilize
the power of multi-core and conquer the load balancing prob-
lem. Furthermore, the parallel programming models research
attempted to optimize the level of parallelism. For instance, the
optimistic parallelism based speculative asynchronous message
passing introduced in 2010, has exploited more parallelism
and reduced the overhead of asynchronous during runtime by
speculating the execution of message passing in object oriented
program [5]. On the other hand, the expanding in parallel
programming models permits the models to be involved in
several domains such as design patterns. According to [6]
they provided a design patterns based parallel programming
model DPBPPM and system implementation in SMP platform
environment. The DPBPPM has been proved to be flexible
and efficient system. There is a development and appreciated
effort on high performance computing research on advance
parallel programming models. For instance, the Center for
Programming Models for Scalable Parallel Computing is a
research center that concentrates on programming models for
scalable parallel computing research area. The research center
is led by Argonne National Laboratory and other Universities,
where they try to develop the current models and advance
models.
III. PARALLEL P ROGRAMMING M ODELS AND T OOLS
In parallel programming models section, presents an ap-
proach of different models of parallel programming. This
section, it describes briefly the different parallel program-
ming models and approaches including Paralliezing Compilers,
Parallel Language, High Performance FORTRAN, Message
Passing, Virtual Shared Memory, Data Parallel, Programming
Skelton and The Partition Global Address Space.
A. Parallelizing Compilers
Paralliezing compiler is a type of source to source compiler,
which its input and output are a high level language. It transfers
the program into parallel output version after analyzing the
sequential program and detecting which part of program can
be parallel. The output program language is usually the same
as the input program language. The most common language
that is used with the paralliezing compiler is FORTRAN. There
are two significant reasons that made FORTRAN language is
the best language that can be used with paralliezing compiler.
First reason, The Fortran is been written in many scientific
computing algorithms. Seconds, analyzing and transforming
the program is easier because FORTRAN has some properties
that it makes paralliezing procedures easy such as static mem-
ory model [7]. The paralliezing compiler works well in some
applications on shared memory multiprocessors. Although,
it has some limitation on other applications that deals with
distributed memory machines, due to non informality time of
access a memory [1].
B. Parallel Language
Most of the people are not willing to use other languages
which they are not familiar with; they would like to use
Authorized licensed use limited to: Nitte Meenakshi Institute of Technology. Downloaded on October 27,2022 at 13:21:27 UTC from IEEE Xplore. Restrictions apply.
their traditional languages such as, C language program and
FORTRAN. Furthermore, the idea behind using the parallel
programming language is hard; make the programmers prefer
using the sequential languages more than parallel languages.
C. High Performance FORTRAN
In 1993, the first High performance FORTRAN version was
published to present effective use of the language on different
parallel systems. High performance FORTRAN is an extension
of Fortran 90; it supports distributing computations of a single
array over multiple processors by using data parallel model. It
does an efficacious performance on applications that follows
(single instruction multiple data), and (multiple instruction
stream multiple data stream) architectures. On the other hand,
the High Performance Fortran needs to be addressed if it is
used with some applications that follow asynchronous struc-
ture.
D. Message passing
Message passing has libraries which designed to be stan-
dard for disturbed memory systems, additionally; it designed to
perform the sending, receiving and other operations of message
passing. The message passing offers a normal synchronization
among processors. One of the beneficial ways of message
passing is performed in the debugging, because it does not
permit for overwriting memory, even if it is happened it can be
detected easily than shared memory [8]. It works efficiency and
naturally in distributed memory systems. The mostly common
systems of message passing are Message Passing Interface and
Virtual Parallel Machine.
1) Message Passing Interface: The Message Passing In-
terface (MPI) where each processor permits to have an ac-
cess by its CPU to the local memory and these processes
can communicate with each other by sending and receiving
messages [9]. Furthermore, the transformation of the data
between these processors is desired to have cooperative op-
erations that match the sending side with its receiving side.
Among concurrent processors the Message Passing Interface
(MPI) provides a communication between them, including
point to point communication and copulative communications.
Moreover, the MPI implementation includes parallel machine,
shared memory machine, distributed memory multiprocessor,
workstations cluster and heterogeneous network. MPI is used
by developer and users when the parallel system relies on
message passing, because its standardization, portability, avail-
ability, performance, additionally, it is supportable by many
HCP platforms.
2) Parallel Virtual Machine: The Parallel Virtual Machine
(PVM) was developed and PVM3 was completed in 1993 [10].
Essentially, it is software that allows of using a collection
of the heterogeneous of UNIX and/ or Windows computers
to be used as single distributed parallel processes. The PVM
system consists of two main parts which are PVM3 that found
with the virtual machine on the computer, and a library of
PVM interface that includes a collection of tools to help the
coordination among tasks. PVM model provides the user the
environment of the heterogeneous with some features [10]:
• Explicit message passing model: each one performs a
computation in multitask set.
1023
 • Heterogeneity support: it is supported by PVM in their
network, applications, and machines.
• Multiprocessor support: it attempts to exploit the un-
derlying hardware by using some methods on multi-
processors.
• Processes based communication: the parallelism item
in PVM is a task that transports among computations
and communications, where each task has to be iden-
tified by unique integer across the whole system.
• User configured host pool: is a set that could in-
clude single-processor and multi-processor computers.
It subjects to adjust or delete during the operations.
The PVM computing model depends on the tasks that are
being a content of the application, where each task performs
a part of the computation. The functionality of these tasks is
various some of them need to be functional in parallel, and
other need to be synchronize, and that specifies any task can
be start or stop and other task may be add or delete during the
executions phase.
E. Virtual Shared Memory
TThe traditional shred memory concept where the mem-
ory is distributed among processors is different than the
virtual shared memory. The virtual shared memory is all
the processors on the distributed memory machine as they
all share a single memory. The objective of designing the
virtual shared memory is reducing the required overhead of the
communication of coherent access. The virtual shared memory
implantation can be accomplished at any level of hierarchical
computer level as shown in Figure 1 [11].
Fig. 1: Computer Hierarchical Levels.
At the hardware level where the virtual machine requires
some special machines such as DASH and DDM, the data size
is fixed number of bytes around 16-256 bytes, but the size is
not located by the hardware level. At the compiler level, the
virtual shared memory can be implemented at this level, where
the data can be larger as it is required. In addition, the compiler
can maintain a set of data where they are logically connected.
The virtual shared memory is also provided at the operating
system level as it is shown in Figure 1 The size of data is
fixed as well as the hardware level, but at this level the data
size is larger than the hardware level. At the high level where
the virtual machine is implemented in the system software is
providing the programmer more flexibility than other levels.
In this level, the data manger process is provided to observe
and control operations.
Authorized licensed use limited to: Nitte Meenakshi Institute of Technology. Downloaded on October 27,2022 at 13:21:27 UTC from IEEE Xplore. Restrictions apply.
F. Data Parallel
Most of the parallel on data parallel model concentrate
on using operations on data which being structured. On each
structure, a group of tasks is being operated, and each task
operates on distance partial of the data structure. The data
parallel model can be used on either SPMD multicomputer or
SIMD computers. In SIMD synchronization, all the processors
are implemented in lockstep fashion. At the compiler time, the
data parallel model’s synchronization can be performed. Using
the data parallel model with SIMD computer makes debugging
and writing the code simple, due to the explicitly of parallelism
that is processed by the flow control, and synchronization
hardware. In general, the data parallel program is easy to
make visualization for the program behavior, additionally; it
has natural load balance [12]. Examples of data parallel are
image processing, N-body problem, and matrix operations.
G. Programming Skelton
It is a collection of high level abstraction where most of
the parallel paradigms are supported. The paradigm for the
programming has the same control structure, and at the same
time can solve different problems. The programming paradigm
contains useful data and communication patterns, additionally;
it contains an abstraction of the form of the programming
or Skelton. A specific parallel programming paradigm is
corresponded by Skelton which has a single abstraction that
encapsulates both the communication patterns and control.
Then, it identifies the parallel programming paradigms. Basi-
cally, Skelton is implemented on the top of shared memory or
distributed memory, message passing, and object oriented. As
a result, Skelton considered as general program that enhance
parallel [1].
H. Partitioned Global Address Space
The processors share common addresses that are obtained
by the partitioned process for the global addresses. The Par-
titioned Global Address Space (PGAS) is described as easy
model to program and understand. It is considered as the
best for both models distributed memory model and shared
memory model. “The PGAS is the best of both worlds. This
parallel programming model combined the performance and
data locality (partitioning) features of distributed memory
with the programmability and data referencing simplicity of a
shared-memory (global address space) model [13]“. The PGAS
programming model seeks to reach these characteristics by
presenting [13]:
• A local view programming style (which identifies
between data partitions whether is local or remote data
partitions).
• A compiler that introduce communication to deter-
mine remote references.
• An address space that can be utilized globally and can
be accessed by any process directly.
• Improving inter-process performance through one-
sided communication.
• Supporting verities of distributed data structure.
1024
 Fig. 2: Parallel Algorithm Design Stages.
Each process has a private and shared memory. For instance,
if processor has a local data, the private memory will be used
for the local data. Similar, if the processor has a global data,
the shred memory will be used for the global data. With single
address, the process can access directly any global data. PGAS
programming languages consists of three main programs that
are widely used: Co-Array FORTRAN (CAF), Unified Parallel
C (UPC), and Titanium [13].
IV. PARALLEL A LGORITHM D ESIGN
There are a verity ways of designing and building a paral-
lel program. The design methodology that we will illustrate
is proposed by Ian Foster that permits the programmer to
concentrate on issues that related to machine independent,
such as concurrency in the early stage, and machine specific
aspects of design are remained until the end of the design
process [1]. The design methodology is identified by four
main stages: agglomeration, communication, mapping and
partitioning. Both partitioning and mapping stages are attempts
to develop the scalability and concurrently algorithms, while
the agglomeration and mapping stages are concentrate more on
locality and other issues that related to performance as shown
in Figure 2.
A. Communication
The computations and the data in which can be decom-
posed into a small tasks. The domain/data decomposition is
a problem that related to the data decomposition, and the
functional decomposition is the decomposition computation
into tasks. There are some issues in portioning step when the
number of computer’s process is ignored.
B. Partitioning
The communication is desired the coordination between
the tasks that are obtained from the partioning phase. The
communication pattern is determined in its phase. There are
four different communication patterns: static or dynamic, struc-
tured or unstructured, local or global, and synchronous or
asynchronous [1], [14].
Authorized licensed use limited to: Nitte Meenakshi Institute of Technology. Downloaded on October 27,2022 at 13:21:27 UTC from IEEE Xplore. Restrictions apply.
C. Agglomeration
The communication structure and the tasks that made dur-
ing the previous phases are evaluated. In order, for improving
performance or reducing the cost, the tasks are grouped into
large group.
D. Mapping
Mapping focuses on assigning each task to a processor,
in order to maximize the processes utilization and minimize
the communication’s cost. Mapping can be executed during
the compiler time (statically) or run time (dynamically) using
load-balancing methods.
V. PARALLEL P ROGRAMMING PARADIGMS
When we classify different parallel applications into pro-
gramming paradigms, we found a few frequently paradigms
that are used in more than one applications. The parallel
programming paradigms includes different algorithms which
based on the paradigms that are used. Selecting the appropriate
paradigms to develop a parallel application is based on the
algorithms in different paradigms. Furthermore, the type of
the availability of some recourse for parallel computing, and
the variety of parallel computers that inherits in the problem,
are all helps to determine which parallel is required.
A. Choice of Paradigms
Most of the authors who, classified the parallel program
into different classes, did not have precisely the same approach.
Although, the following paradigms are the most popular
paradigms that are used on the parallel programming: Task
Farming, Single Program Multiple Data, Divide and Conquer,
Data Pipelining, Hybrid models, and Speculative Parallelism
[15].
B. Task Farming
The Task Farming paradigms contains two major processes.
First, the master process handles the decomposing of the
problem into several tasks, and then distributes these tasks into
1025
a farm of slave process. Subsequently, after finishing the slave
process, the master assembles partial result in order to compute
the final result. Second, the multiple slave process execute in a
small cycle: getting the message with the task, processing the
task and sending the result to the master. The communication
in Task Farming usually occurs among the master process and
slave process [15].
This paradigm can be used in either static load-balancing
or dynamic load-balancing. In the static Load-balancing, the
distribution tasks appear in the first stag. As a result of tasks
distribution, the master can be contributed in the computation,
after reserving the slave. In the dynamic load-balancing, if the
numbers of tasks are larger than the number of processes, it
can be more appropriate. The dynamic has a significant feature
where the application can alter the system and recognize
the resources of the system. In general, the Task Farming
paradigms can deliver a high scalability and speedily level of
computation [1].
C. Single Program Multiple Data
In the Single Program Multiple Data (SPMD), divides the
data between the processors, and each processor runs a part
from the total basic algorithm on the data. Finally, it will gather
the final result in the end.
This kind of parallelism can be a geometric parallelism
or domain decomposition or data parallelism. The SPMD is
the most widespread paradigm is used. This paradigm can be
performed efficiently, if the data distributed equally among the
processes and the system is homogenous [1], [15].
D. Data Pipelining
In the data pipelining, is one of the most common de-
composition processing paradigm, due to its simplicity and
robustness. It identifies the parallel tasks on the algorithm
where each processor runs a part of the algorithm. The data
flows in the pipeline from one processor which corresponds to
a phase in the pipeline structure to another processor until the
last processor that provides the output. Each process executes
a fraction of the algorithm, and these processes communicate
with each other through data flowing. Usually, the data pipeline
paradigm is used on the image processing and data reduction
application [1], [15].
E. Divide and Conquer
The divide and conquer is a top-down approach. The
solution of main problem where is in the top level (main
problem) is obtained by combining the solutions of the sub
problems in the down levels. It divides the main problem into
simple sub problems, and sometime decomposes these sub
problems into more simple problems if it possible, until each
sub problem has a simple solution. Then, it gathers all the sub
problems’ results to obtain the solution for the main problem.
Giving sufficient parallelism, the sub problems can be solved
concurrently.
The structure of the divide and conquer algorithm is
perform as a tree where the main problem is the root and the
sub problems are the nodes. The dividing and combination of
the sub problems to obtain the main problem formulate uses
Authorized licensed use limited to: Nitte Meenakshi Institute of Technology. Downloaded on October 27,2022 at 13:21:27 UTC from IEEE Xplore. Restrictions apply.
some parallelism, although the processors require few com-
munication process. There is no required for communication
processes between sub processes, because the sub problems
are independent [1]. Basically, the divide and conquer can be
performed into three main steps: compute, spilt and join.
F. Speculative Parallelism
If the parallelism is hard to be obtained through one of
the pervious paradigms, in this situation the Speculative Par-
allelism is being used. Even though, there are some problems
cannot achieve the parallelism, due to the data dependencies
complex. The speculative parallelism uses some optimistic
operation to simplify the parallelism and executing the problem
in small fractions. Some cases of using the Speculative Paral-
lelism, are discrete event simulation (asynchronous problem),
and using different algorithms for the same problem (the first
one is the one provide the final solution) [1], [15].
G. Hybrid Models
The Hybrid Models is called Mixed Mode Programming;
it consists of more than one paradigm. The Hybrid Models is
used when the mix element of different paradigms is required.
An example of Hybrid model is OpenMP with MPI, where the
message passing with MPI is used for a communication, and
the OpenMP for each single node is used to control a thread.
VI. PARALLEL P ROGRAMMING
There are many existing parallel programming. This section
will discuss and compare between the widely known paral-
lel programming including OpenMP, and Threading Building
Block in terms of many aspects.
A. OpenMP
OpenMP, which stands for Open Multi-Processing, was
developed in cooperative efforts of several software developers
and companies manufacturing, such as Oracle, IBM, Intel, and
Hewlett-Packard. It is an Application Program Interface (API)
that supports multi-platform shared memory parallel program-
ming: C, C++ and FORTRAN [16]. OpenMP could be found
in most of the mentioned language architectures including
UNIX platform, and Windows NT platform. The OpenMP
provides the shared-memory parallel programmers a simple
interface for developing parallel application, and that due to
OpenMP portability and scalability. The OpenMP portability
is a significant characteristic. It permits the compiler, that sup-
portable by OpenMP, to be used in any parallel application that
developed by OpenMP. For attaining the parallel performance,
the compiled binary result must be executed on the hardware
platform [17].
The following OpenMP is written on C++, and FORTRAN
[17]. In the example, the loop adds the Y array to the X array
where are executed in parallel.
Another OpenMP example
This example demonstrates the using of Wordcount program
with OpenMP. The Wordcount is a program that counts the
number of occurrence unique word in each read file. After the
program reads most of the files, it will show the whole number
of the unique words and the most frequent word in the files. In
1026
the file, each line can be split between several processors that
provide sufficient parallelism to speed up the word counting.
The processes’ results unite to deliver the final result.
The input program will be a file which could be quite large,
because of that the file will be parsed in parallel. In order, this
program needs to create an object of the following classes:

• Parser obj: it is used to parse multiple lines of the file

into strings.
• Collect obj:it is used to collect and combine the
results that are produced by Parser objects.

The Wordcount program will be used with the OpenMP

as shown in Figure 4. It features for executing the while
loop in parallel, and that will accomplish the parse phase.
Consequently, using the parallel for directive, the collect phase
will be operated. The OpenMP feature will be used to count
the number of different occurrence word and the whole number
of words [17]. OpenMP provides the user the parallelism
that is marked in the loop and other part of the code. The
OpenMP has the advantage where is easy to transfer the Fig. 3: TBB WordCount program [17].
process from sequential program into parallel OpenMP pro-
gram. The OpenMP scalability is restricted by the number of
process cores and the complexity of the computational logic.
In general, the OpenMP presents a standard between diversity
of shared memory architectures, and creates a set of directive
for a programming of shared memory machines. In addition,
the OpenMP is easy to use by providing the capability of
incrementing parallelize a serial program, and the capability
of implementing fine-grain and coarse-grain parallelism [16].
It supports the following language C, C++ and FORTRAN
(77, 90, and 95). Finally, The OpenMP 3.0 is a new version
that was developed in 2008; it makes the parallel programming
model more flexible by providing a dynamic task

B. Threading Building Block

The Threading Building Block (TBB) presents an approach
of expressing the parallelism in C++ program. It is a C++ tem-
plate library that includes an implementation for some parallel
algorithms, such as Parallel for, Parallel while, Paralle reduce
and more. Furthermore, the TBB “reduce the developer burden
of managing complex and error-prone parallel data structure”
by offering a set of data structure parallel, such as Concr-
rent queue, Concurrent vector, and Concurrent hash [17]. The
TBB has a central execution unit that can be scheduled by the
library’s run-time engine.
An Example of Threading Building Block is the Word-
Count program illustrated in Figure 3. It is including the pervi-
ous classes (Parser obj and Collect obj) will be used again in
this example with TBB, consequently, it will exploit numerous
features in TBB. By Using Parallel for, each thread will be
operated on a chunk that contains the broken iteration space.
A Blocked range abstracts the one-dimensional space. The
ApplyParser class is used the parallel for by invoking it from
the main program. The TBB permits the appropriate reduction
operation in parallel. Another class called ReduceCollect Class
shows the characteristics of a reduction operation. For each
element in the block, the Operator () identifies the iterative
operation. A reduction object is required [17].
Fig. 4: OpenMP WordCount program [17].
According to [8], in 2011 Intel adopted Open CASCADE
with Intel TBB. The TBB is being selected for Open CAS-
CADE, because TBB provides the scalable memory allocated,
parallel algorithms and simplicity of integration. The BBT
is one of the best shared memory parallel programming that
form on run time library. It is aimed to reduce the effort that
program is needed to write parallel program. The TBB presents
a protection for the developer from the precise differences of
thread management, and the programming error. In contrast,
comparing the TTB with OpenMP, the OpenMP is easier
for converting from sequential program into parallel program.
Furthermore, creating a program is still complicated in TBB
[17].
VII. I MPLEMENTATION AND E VALUATION
In order to evaluate OpenMP parallelism, we use the
matrix multiplication scenario as it delivers the best case
for parallelization. Matrix multiplication algorithm computes

1027

Authorized licensed use limited to: Nitte Meenakshi Institute of Technology. Downloaded on October 27,2022 at 13:21:27 UTC from IEEE Xplore. Restrictions apply.
 is supported by OpenMP, while TBB provides similar data
access locks called mutex. Finally, TBB does not require
any special language or compiler unlike OpenMP that uses
program directives for compiler [18], [19].

TABLE I: Comparison between OpenMP and TBB.

Feature OpenMP TBB
Offloading + -
Static scheduling + -
Parallelism complexity - +
Nested parallelism support - +
Abstraction of memory hierarchy + -
Mutual exclusion + -
Not required compiler - +
Fig. 5: OpenMP Speedup.

the product of two input matrices. The algorithm operation
carries out by three main nested loops, where two loops for
initialization and the other one for multiplication. The matrix
multiplication algorithm was tested in dual-core Intel Core(R)
N4000 CPU, 1.10 GHz system with RAM of size 4 GB. We
evaluate the OpenMP parallelism model in terms of speedup
and efficiency. As shown in Figure 5, the time decreeses as the
number of threads grow. Similar, the efficiency of the model
rises with the increasing number of threads in Figure 6.
VIII. C ONCLUSIONS
In summary, the programmers are able to express the
parallelism aspect in their programs while concurrently exploit
the underlying hardware architecture capabilities by using
different parallel programming models. Each parallel program-
ming model has different way of exploiting parallelism. Cur-
rently, the parallel programming models research area previews
advance development between whiles. We believe during a
few years the parallel programming performance, will be ten
times than it is including the current and new models. This
paper describes briefly diverse parallel programming models
including Paralliezing Compilers, Parallel Language, High
Performance FORTRAN, Message Passing, Virtual Shared
Memory, Data Parallel, Programming Skelton and The Parti-
tion Global Address Space. Furthermore, it explains the most
common paradigms: Task Farming, Single Program Multiple
Data, Divide and Conquer, Data Pipelining, Hybrid models,
and Speculative Parallelism. Finally, it discusses the parallel
design algorithms, and illustrates some examples for parallel
programming models.

[1]
Fig. 6: OpenMP Efficiency. [2]
[3]
The closer model to OpenMP is TBB, thus a comparison
between the two models have been made. To compare OpenMP
[4]
and TBB based on criteria of our interest, we summaries
the most significant differences between both models. As
illustrated in Table I, offloading feature is the parallelism [5]
process between both the host and device that mainly support
the acceleratorbased system. OpenMP supports the offloading
between both host and target device, while TBB only support
offloading on host side. Another feature is static scheduling [6]
where threads execution order can be controlled. The execution
order is supported by OpenMP while it is missed in TBB.
OpenMP does not support nested and complex parallel patterns [7]
as in TBB. In comparison, OpenMP provides construct mem- [8]
ory hierarchy for programmers to specify memory locations.
Mutual exclusion for protecting data access in parallel program
R EFERENCES
R. Buyya et al., “High performance cluster computing: Architectures
and systems (volume 1),” Prentice Hall, Upper SaddleRiver, NJ, USA,
vol. 1, no. 999, p. 29, 1999.
B. Barney et al., “Introduction to parallel computing,” Lawrence Liv-
ermore National Laboratory, vol. 6, no. 13, p. 10, 2010.
W. Giloi, “Parallel programming models and their interdependence with
parallel architectures,” in Proceedings of Workshop on Programming
Models for Massively Parallel Computers. IEEE, 1993, pp. 2–11.
J. Peng, C. Hu, and J. Xi, “Msi a new parallel programming model,”
in 2009 WRI World Congress on Software Engineering, vol. 1. IEEE,
2009, pp. 56–60.
Y. Du, Y. Zhao, B. Han, and Y. Li, “Optimistic parallelism based on
speculative asynchronous messages passing,” in International Sympo-
sium on Parallel and Distributed Processing with Applications. IEEE,
2010, pp. 382–391.
H. Wu, “Design-pattern based parallel programming model and system
implementation,” in 2008 4th International Conference on Wireless
Communications, Networking and Mobile Computing. IEEE, 2008,
pp. 1–5.
F. Plavec, “Dependence testing for parallelizing compilers,” 2003.
Intel., “oneapi threading building blocks,”
https://software.intel.com/content/www/us/en/develop/tools
/oneapi/components/onetbb.html.

1028

Authorized licensed use limited to: Nitte Meenakshi Institute of Technology. Downloaded on October 27,2022 at 13:21:27 UTC from IEEE Xplore. Restrictions apply.
 [9] M. H. P. C. C. SP Parallel Programming Workshop, “Message passing [15] K. P. Kenneth Pedersen, “Scientific applications in distributed systems,”
interface,” http://www.mhpcc.edu/training/workshop20/mpi/MAIN.html. http://www.idi.ntnu.no/grupper/su/sif8094-reports/2001/p7.pdf, 2001.
[10] S. H. Roosta, Parallel processing and parallel algorithms: theory and [16] “The openmp api specification for parallel programming,”
computation. Springer Science & Business Media, 2012. https://www.openmp.org//.
[11] A. Chalmers, E. Reinhard, and T. Davis, Practical parallel rendering. [17] Oracle, “Developing parallel programs - a discussion of
CRC Press, 2002. popular models,” https://www.oracle.com/technetwork/server-
[12] M. Parmar, “Data parallel model and object oriented storage/solarisstudio/documentation/oss-parallel-programs-170709.pdf,
model,” http://www.gazhoo.com/doc/201006110254229749/DATA- 2016.
PARALLEL+MODEL, 2009. [18] E. Ajkunic, H. Fatkic, E. Omerovic, K. Talic, and N. Nosovic, “A
[13] R. Galal, “Partitioned global address space (pgas),” comparison of five parallel programming models for c++,” in 2012
https://www.mohamedfahmed.wordpress.com/2010/05/06/partitioned- Proceedings of the 35th International Convention MIPRO. IEEE, 2012,
globaladdress-space-pgas/, 2010. pp. 1780–1784.
[14] I. Foster, “Designing parallel algorithms,” [19] S. Salehian, J. Liu, and Y. Yan, “Comparison of threading programming
https://www.mcs.anl.gov/ itf/dbpp/text/node14.html, 1995. models,” in 2017 IEEE International Parallel and Distributed Process-
ing Symposium Workshops (IPDPSW). IEEE, 2017, pp. 766–774.

1029

Authorized licensed use limited to: Nitte Meenakshi Institute of Technology. Downloaded on October 27,2022 at 13:21:27 UTC from IEEE Xplore. Restrictions apply.