2020 IEEE International Conference on Autonomic Computing and Self-Organizing Systems Companion (ACSOS-C)

Evaluating performance of Parallel Matrix Multiplication

Routine on Intel KNL and Xeon Scalable Processors

Thi My Tuyen Nguyen Yoosang Park Jaeyoung Choi Raehyun Kim

School of Computer Science and
engineering
Soongsil University
Seoul, South Korea
nguyentuyen0406@soongsil.ac.kr
School of Computer Science and
Engineering
Soongsil University
Seoul, South Korea
yspark@soongsil.ac.kr
School of Computer Science and
Engineering
Soongsil University
Seoul, South Korea
choi@ssu.ac.kr
Department of Mathematics
University of California, Berkeley
Berkeley, US
rhkim79@berkeley.edu

Abstract— In high-performance computing, xGEMM
routine is the core of Level 3 BLAS operation to achieve
matrix-matrix multiplications. The performance of Parallel
xGEMM (PxGEMM) is significantly affected by two major
factors: Firstly, the flop rate that can be achieved by calculating
matrix-matrix multiplication on each node. Secondly,
communication costs for broadcasting sub-matrices to others.
In this paper, an approach to improve and adjust PDGEMM
routine for modern Intel computers: Knights Landing (KNL)
and Xeon Scalable Processors (SKL) is proposed. This
approach consists of two methods to deal with the factors
mentioned above. First, the improvement of PDGEMM for the
computation part is suggested based on a blocked matrix-matrix
multiplication algorithm by providing better fits for
architectures of KNL and SKL to deliver a better block size
computation. Second, a communication routine with MPI is
proposed to overcome default settings of BLACS which is a
part of communication, to improve a time-wise cost efficiency.
The proposed PDGEMM achieves similar performance on
smaller size matrices as PDGEMM from ScaLAPACK and
Intel MKL on 16 node Intel KNL. Furthermore, the proposed
PDGEMM achieves better performance (on smaller size
matrices) compared to PDGEMM from ScaLAPACK and Intel
MKL on 16 nodes Xeon scalable processors.
Keywords— Parallel matrix-matrix multiplication, Parallel
BLAS, ScaLAPACK, Intel Knights Landing, Intel Xeon Scalable,
AVX-512
I. INTRODUCTION
Over the last decade, the description of what is called high-
performance computing has altered dramatically. In 1988, an
article published in Wall Street Journal titled “Attack of the
Killer Micros” described that how computing systems made up
of many small inexpensive processors would soon make large
supercomputers obsolete. This vision has come true in some
ways, but not in the way the original proponents of the “killer
micro” theory envisioned. High-performance computing runs
an extensive range of systems, from our desktop computers
through large parallel processing systems. The study of high-
performance computing provides a chance to revisit computer
architecture and enhance it for achieving improvement in their
performance.
High-performance computing systems are progressively rising
interest to provide new levels of computational capability for
an increasing range of applications. For the past few years, as a
software developer, we have had the opportunity to observe the
development of computational software (high-performance
computing libraries) that are built for high-performance
computing (HPC) [1].
These libraries are used as design tools and building blocks for
engineers, developers, and researchers in many different areas.
In recent years, numerical libraries for parallel computations
are available, and have become a standard in the computing
community. It includes Basic Linear Algebra Subprograms
(BLAS) [2], Parallel BLAS (PBLAS) [3], Parallel Engineering
and Scientific Subroutine Library (PESSL) [4], Scalable Linear
Algebra Package (ScaLAPACK) [5], and Intel Math Kernel
Library (MKL) [6]. PBLAS is a part of the ScaLAPACK library.
Besides that, Parallel General Matrix-Matrix Multiplication
(PxGEMM) is an important routine for high-performance
computing, which is included in ScaLAPACK/PBLAS,
PESSL/PBLAS, and Intel MKL, however, the performance
varies respectively. The PxGEMM routines are available in
several precision types: single (S), double (D), complex (C),
and double complex (Z) precisions. In this paper, the parallel
double-precision general matrix-matrix multiplication
(PDGEMM) is considered. Because the source code of
PDGEMM routine from MKL is not provided and it only
provides as in binaries that is compatible with every Intel
processor. On the other hand, the performance of PDGEMM
from PESSL is not as good as PDGEMM from ScaLAPACK
and PESSL supports parallel processing applications on IBM
UNIX SP Systems, and clusters of IBM UNIX servers.
Therefore, this paper presents a method for optimizing the
performance of PDGEMM from ScaLAPACK for special Intel
Knights Landing (KNL) and Xeon scalable (SKL) processors.

978-1-7281-8414-2/20/$31.00 ©2020 IEEE 42

DOI 10.1109/ACSOS-C51401.2020.00027

Authorized licensed use limited to: Soongsil University. Downloaded on May 01,2021 at 05:40:50 UTC from IEEE Xplore. Restrictions apply.
 The main contribution of this paper is to improve the
performance of PDGEMM by considering the following
constraints on existing PDGEMM:
(i) Improving the performance of local matrix-matrix
multiplication at each node by using a blocked matrix-
matrix algorithm and setting the size of blocked
computation different from the default one.
(ii) Reduce communication costs by replacing the BLACS
library to MPI library.
II. RELATED WORK
An important part of parallel computing is the mechanism for
transmitting data between nodes. Parallel Virtual Machine
(PVM) [7] and Message Passing Interface (MPI) [8] are
commonly used libraries for evaluating and optimizing
computation power in a parallel environment. PVM provides a
portable heterogeneous environment for using a cluster of
machines using socket communication over TCP/IP as a
parallel computer. MPI is a message passing interface for a
distributed memory system, which enables program portability
among different parallel machines. However, the performance
and losses gains through parallel processing using MPI and
PVM are different. By evaluating performance on a cluster
based parallel computing architecture, the performance of MPI
achieved faster performance compared to PVM [9]. Therefore,
MPI is suitable for providing a message-passing layer for Basic
Linear Algebra Communication Subprograms BLACS [10], a
message-passing library designed for linear algebra [11]. On
the other hand, BLACS also forms the basic communication
layer for the ScaLAPACK and PESSL.
ScaLAPACK is a set of libraries for high-performance linear
algebra routines on a distributed-memory computer system.
PESSL is a scalable mathematical subroutine library. Both
PESSL and ScaLAPACK includes PBLAS. PBLAS performs
global computation of parallel computing, use BLAS for
computation in a process and communication between
processes with BLACS. PBLAS also provides the same three
levels of implementation as BLAS. Besides that, the
performance of PBLAS level 3, PDGEMM in ScaLAPACK is
better than that of PESSL [12]. On the other hand, Intel MKL
is an optimized library for Intel processors, therefore the
performance of PDGEMM from Intel MKL is better than
ScaLAPACK. However, Intel MKL is not officially revealed in
the source code level, therefore, this paper proposes a method
to optimize the performance of PDGEMM from ScaLAPACK
and compare it with MKL on KNL and SKL.
The performance of PDGEMM is significantly affected by
two major factors. Firstly, the flop rate that can be achieved by
calculating matrix-matrix multiplication on each node.
Secondly, communication cost for broadcasting sub-matrices to
other nodes. The performance of PDGEMM is affected by the
performance of the sequential DGEMM routine in each node.
There is a way to choose BLAS libraries for DGEMM such as
OpenBLAS [13], ATLAS [14], BLIS [15], and Intel MKL
BLAS. However, our previous works [16, 17], proposed
DGEMM, which achieves better performance compared to
43
Authorized licensed use limited to: Soongsil University. Downloaded on May 01,2021 at 05:40:50 UTC from IEEE Xplore. Restrictions apply.
DGEMM from OpenBLAS, ATLAS, and BLIS on both KNL,
and SKL. Additionally, the performance of proposed DGEMM
achieves 90% compared to the DGEMM from Intel MKL
BLAS on KNL and on SKL, the performance of proposed
DGEMM is better than that of Intel MKL when matrix size is
less than 6,000. Besides that, the performance of the computing
matrix multiplication at each node also depends on the block
size of the matrix, details are presented in the next section.
Secondly, a method of communication between nodes also
affects the performance of PDGEMM. As mentioned above,
BLACS is used for communicating data in ScaLAPACK.
However, PDEGMM is written in C, but with Fortran 77
interfaces. Besides that, BLACS requires data packaging before
utilizing MPI for communication. Therefore, PDGEMM uses
BLACS for communication that is ineffective than using MPI
directly. Therefore, PDGEMM uses MPI that will help to
reduce idle time occurrences for interconnected processors
during the progress of the communication phase.
III. OPTIMIZATION STRATEGIES
A. High-performance DGEMM routine
High-performance DGEMM routines are essential to
improve the performance of the PDGEMM routine. We have
implemented DGEMM routine based on a blocked matrix-
matrix multiplication algorithm proposed by Goto and van de
Geijn [18] and Gunnels et al. [19]. By blocking the matrices, it
is possible to maximize the data reusability while minimizing
the communication cost. To optimize the proposed DGEMM
routines, we focus on two main factors, the kernel
implementation and the parameter optimization.
To implement the DGEMM routine, we need to build a
packing kernel that packs the sub-matrices into a contiguous
array and an inner kernel that performs the matrix-matrix
computation between the sub-blocks of the source matrices.
Since both kernels have a significant impact on the overall
performance, we exploit the Intel AVX-512 instructions for
both kernels.
The performance of our DGEMM routines highly depend on
the register block sizes, the cache block sizes, the prefetching
distances, the loop unrolling depth and the parallelization
scheme. As we observed in our previous study, the optimal
blocking parameters and memory usage can be different despite
having the same memory structure. So, the auto-tuning
approach can be used to search the optimal parameters. Hence,
we use the auto-tuner in the previous work [20] to generate
optimal DGEMM routines for the given systems.
In addition, there is a critical issue related to the nested-loop
parallelization. In the other previous study [21], we observed
that the relatively low performance caused by the
parallelization scheme and the thread affinity. This
performance loss happens in general, because of the inefficient
cache use. If we do not consider the cache structure of the target
machine, especially the shared cache, then it is difficult to
achieve the optimal cache usage. Therefore, if the given system
has a cache shared by multiple threads, then the number of
threads is bound to the corresponding loops in the nested loop
that is needed to be divisible by the numbers of threads
allocated. Moreover, we need to assure that the logical thread
teams are properly placed along with the physical cores.
Unlike the assumption in our previous study [16-17], the
source matrices of PDGEMM routine are stored in the column-
major order. So, we search the optimal parameters and generate
our proposed DGEMM routine for the column-major case. We
achieve a similar performance against the one with row-major
order on KNL and SKL environments in the same way of
iterating matrices.
B. Inter-nodes communication
1. Blocked matrix computation and distribution
The performance of PDGEMM routine from MKL gives
better performance compared to the PDGEMM from
ScaLAPACK, however, MKL provides compatibility for every
Intel processor. Therefore, the blocking factor KB can be set
44
Authorized licensed use limited to: Soongsil University. Downloaded on May 01,2021 at 05:40:50 UTC from IEEE Xplore. Restrictions apply.
differently depending on computer architectures such as KNL
and SKL. Figure 1 shows the performance of PDGEMM with
different matrix sizes on the corresponding computer
architectures for KNL and SKL. The tested size of matrix is set
to 60,000 for 4 nodes, and is set to 144,000 for 16 nodes. The
better performance of PDGEMM can be achieved when the KB
value is set to 336 for KNL and it is shown in Figure 1.(a) and
(b). Moreover, Figure 1.(c) and (d) show that the better result
of the proposed PDGEMM can be achieved when KB is set to
384 for SKL. Therefore, the good performance of PDGEMM
can be achieved when KBs are applied for various computer
architecture kinds. These blocking factors are commonly used
in not only PDGEMM but also in LU factorization [22].
2. Communication cost
The other impact has been found during the processes. That is,
the communication method may affect performance. For
example, the matrix multiplication algorithm performs a large
number of broadcasting operations on blocked matrices (or sub-
matrices). As long as MPI routines are used to communicate
between nodes, it needs to pack the data into buffers before
sending the data. PDGEMM originally calls the collective
BLACS C routines such as Cgebr2d or Cgebs2d, to broadcast
sub-matrices through the nodes. They pack the data with
BLACS routines and call MPI routines to communicate
between the processors. Therefore, we focused on optimizing
both packing and broadcasting operations.
 First, we implemented the packing kernel using the Intel Cilk
Plus library. It shows better performance than the BLACS
routines. However, the packing only takes up to 4% of the
whole communication time, the main progress was made in the
improvement of the broadcasting routine. Rather than using the
MPI broadcasting routine directly, we developed our
communication routine to broadcast the data by following the
ring topology for both the columns and rows. By
communicating as an asynchronous fashion, we can maximize
the effect of overlapping communication and computation.
Furthermore, we can reduce a large overhead that is observed
in the BLACS routine of both Intel MKL and netlib [23]. Figure
2. shows the impact for communication method with different
matrix size and block size KB=336 on 4 nodes KNL, and
KB=384 on 4 nodes SKL. In Figure 2, the lower is better since
it shows the elapsed time.
IV. EXPERIMENTS
In order to perform the experiment of this study, two scalable
environments are utilized to compare the performance of
PDGEMM from applying a combination of using MKL and
ScaLAPACK. Experiments are performed by utilizing the
facilities provided by the institute named: Korea Institute of
Science and Technology Information (KISTI) [24]. There are
two scalable environments of KNL and SKL which are
provided by KISTI. For KNL, configuration for hardware is set
as cache mode for memory mode, and quadrant mode for
cluster mode. The advantage of testing with these environments
is that both target machines provide the Intel AVX-512
instructions. Table 1 shows a summary of the target machine’s
architecture.
Table 1. Architecture Summary of target Machines
The rest settings of proposed PDGEMM is integrated with
ScaLAPACK version 2.0.2, BLAS from Intel MKL, and Intel
MPI 19.0.5 from Intel Parallel Studio 2019 Update 5 for both
systems. Three test applications are compared: the proposed
PDGEMM fixed from ScaLAPACK that utilizes blocked
matrix-matrix multiplication at each node for the first one, with
the PDGEMM fixed from ScaLAPACK that uses DGEMM
from Intel MKL BLAS for the calculation matrix multiplication
at each node, and the simple application based on MKL which
calls PDGEMM routine.
For executing PDGEMM, each global matrix that is to be
distributed across the process grid must be assigned as an array
descriptor. This array descriptor is called descinit_():
45
Authorized licensed use limited to: Soongsil University. Downloaded on May 01,2021 at 05:40:50 UTC from IEEE Xplore. Restrictions apply.
GHVFLQLWB'(6&0.0%.%565&&65&,&7;70;//'$,1)2
where DESC is the array descriptor of a distributed matrix to be
set; M is the number of rows in the distributed matrix; K is the
number of columns in the distributed matrix; MB and KB are
blocking factors that are used for distributing both rows and
columns of matrices; RSRC and CSRC are the process
indicators how rows and columns should be distributed,
respectively; ICTXT is the BLACS context handler, indicating
the global context of the operation on the matrix; MXLLDA is
the leading dimension of the local array storing the local blocks
of distributed matrix. In PDGEMM from MKL, the good value
to check performance is MB=KB=1,000. Table 2 demonstrates
the three environments of PDGEMM applied:
ƒ The proposed PDGEMM compiled with ScaLAPACK by
using blocked matrix-matrix multiplication at each node,
communication data via MPI routine, using high-
bandwidth on KNL, and blocking factors KB are set as
336 for KNL and as 384 for SKL. -ķ
ƒ PDGEMM environment on ScaLAPACK with using
similar configurations in the proposed PDGEMM but
using DGEMM from Intel MKL BLAS for calculation
matrix at each node. -ĸ
ƒ PDGEMM routine from built-in MKL -Ĺ
1. Experiment results for System 1 (KNL)
The Intel Xeon Phi processor supports high-bandwidth
memory (HBM). HBM is faster up to five times (൒ 400 GB/s)
compared to DDR4 memory (൒90 GB/s). Buffer memory of
blocked matrix A and B are replaced by using HBM to evaluate
the performance of proposed PDGEMM on System 1 using
memkind library as long as the matrix size is acceptable to be
stored.
We demonstrate the performance of the KNL environment in
Figure 3. Figure 3.(a) and (b) show the results from 4 nodes (P
h Q=2 h 2) and 16 nodes (P h Q=4 h 4), respectively. The
parameters for these experiments are set as KB=336 for
environments ķ and ĸ and K =1,000 for PDGEMM routine
from MKL, where M=N=K. The PDGEMM from the proposed
environment ķ shows higher performance as compared to Ĺ
i.e. 25% on 4 nodes for M=N=K=12,000 and 63% on 16 nodes
for M=N=K=20,000. On the other hand, if we compare the
results of the proposed environment ཛ with environment ĸ,
the performance seems to be similar when the matrix size is
smaller than 20,000 on 16 nodes, and the performance of the
proposed environment ķ shows 70% better performance
compared with the environment ĸ on 4 nodes.

2. Experiment results for System 2 (SKL)
For SKL, the results are shown in Figure 4. It shows the
results from 4 nodes (P×Q=2×2) and 16 nodes (P×Q=4×4),
respectively. The parameter of KB for this experiment is set to
384 for environment ķ and ĸ whereas KB=1,000 for
environment Ĺ, and M=N=K. The results from Figure 4.(a)
shows that the environment ķ achieved 61% better
performance compared to environment Ĺ on 4 nodes SKL at
matrix size of M=N=K=24,000. Furthermore, Figure 4.(b)
shows that the environment ķ achieved 75% better
performance compared to the environment Ĺ on 16 nodes
where the matrix size is set to M=N=K=20,000. On the other
hand, environment ķ achieves better performance compared to
the environment ĸ on small size matrices around 8,000 on 4
nodes SKL. Apart from 4 nodes, the environment ķ achieves
98% similar performance as environment ĸ on 16 node SKL
at a matrix size of 20,000.
V. CONCLUSIONS
In this paper, we presented a method to improve the
performance of PDGEMM routine. The proposed approach
contains two improvement factors, computation and inter-node
communication for modern scalable processors on KNL and
SKL. The proposed PDGEMM achieves better performance
than the PDGEMM from ScaLAPACK for both KNL and SKL
with relatively small size of matrices. For future work, the
suggested approach will be adjusted for the further dense matrix
multiplication routines such as LU, QR, and Cholesky
factorization.
ACKNOWLEDGEMENT
This work was supported by the Next-Generation
Information Computing Development Program through the
National Research Foundation of Korea (NRF) funded by the
Korea government (MSIT) (NRF-2015M3C4A7065662).

46

Authorized licensed use limited to: Soongsil University. Downloaded on May 01,2021 at 05:40:50 UTC from IEEE Xplore. Restrictions apply.
 REFERENCES
[1] “High-Performance Computing (HPC),” Available :
https://www.nics.tennessee.edu/computing-resources/what-is-hpc/.
[2] BLAS. Available: http://www.netlib.org/blas/.
[3] PBLAS. Available: http://www.netlib.org/pblas/.
[4] S. Filippone, “Parallel libraries on distributed memory architectures: The
IBM Parallel ESSL,” in Applied Parallel Computing Industrial
Computation and Optimization, -:DĞQLHZVNL-'RQJDUUD.0DGVHQ,
and D. Olesen Eds. Springer, 1996.
[5] ScaLAPACK. Available: http://www.netlib.org/scalapack/.
[6] Intel MKL. Available:
https://software.intel.com/content/www/us/en/develop/tools/math-
kernel-library.html/.
[7] A. Geist, A. Beguelin, J. Dongarra, W. Jiang, R. Manchek, and V.
Sunderam, “PVM: Parallel virtual machine: a users' guide and tutorial for
networked parallel computing,” MIT Press, Cambridge, MA, 1994.
[8] R. Hempel, “The MPI standard for message passing,” in High-
Performance Computing and Networking, W. Gentzsch and U. Harms
Eds. Springer, 1994, pp. 247-252.
[9] S. Sampath, B. B. Sagar, and B. R. Nanjesh, “Performance evaluation and
comparison of MPI and PVM using a cluster based parallel computing
architecture,” 2013 International Conference on Circuits, Power and
Computing Technologies (ICCPCT), Nagercoil, pp. 1253-1258, 2013.
[10] BLACS. Available: http://www.netlib.org/blacs/.
[11] D. Walker, W. Sawyer, and V. Deshpande, “An MPI Implementation of
the BLACS,” in MPI Developers Conference, Notre Dame - South Bend,
vol. 1, pp. 0195, 1996.
[12] K. Petter and S. Patrik Skogqvist, “BLACS, PBLAS, PESSL and
ScaLAPACK Libraries on the IBM SP,” 1997. BLAS BLACS, 2007.
[13] X. Zhang, Q. Wang, and S. Werber, Openblas. Available:
http://www.openblas.net/.
[14] R. C. Whaley, A. Petitet, and J. Dongarra, “Automated empirical
optimizations of software and the atlas project,” Parallel Computing, vol.
27, 2001, pp. 3–35, doi: 10.1016/S0167-8191(00)00087-9.
47
Authorized licensed use limited to: Soongsil University. Downloaded on May 01,2021 at 05:40:50 UTC from IEEE Xplore. Restrictions apply.
[15] F. G. Van Zee and R. A. Van De Geijn, “BLIS: a framework forrapidly
instantiating BLAS functionality,” ACM Transactions on Mathematical
Software, vol. 41, 2015, pp. 14, doi: 10.1145/2764454.
[16] R. Lim, Y. Lee, R. Kim, and J. Choi, “An implementation of matrix-
matrix multiplication on the Intel KNL processor with AVX-512,”
Cluster Computing, vol. 21, 2018, pp. 1785–1795, doi: 10.1007/s10586-
018-2810-y.
[17] R. Lim, Y. Lee, R. Kim, J. Choi, and M. Lee, “Auto-tuning GEMM
kernels on the Intel KNL and Intel Skylake-SP processors,” Journal of
Supercomputing, vol. 75, 2019, pp. 7895-7908, doi: 10.1007/s11227-018-
2702-1.
[18] K. Goto and R. A. van de Geijn, “Anatomy of high-performance matrix
multiplication,” ACM Transactions on Mathematical Software (TOMS),
vol. 34, 2008, pp. 1-25, doi: 10.1145/1356052.1356053.
[19] J. A. Gunnels, G. M. Henry, and R. A. Van De Geijn, “A family of high-
performance matrix multiplication algorithms,” International Conference
on Computational Science, Springer, 2001, doi: 0.1007/3-540-45545-
0_15.
[20] R. Kim, J. Choi, and M. Lee, “Optimizing parallel GEMM routines using
auto-tuning with Intel AVX-512,” Proc. International Conference on
High Performance Computing in Asia-Pacific Region, 2019, doi:
10.1145/3293320.3293334.
[21] R. Lim, Y. Lee, R. Kim, and J. Choi, “OpenMP-based parallel
implementation of matrix-matrix multiplication on the intel knights
landing.” Proc. Workshops of HPC Asia, 2018, pp. 63-66, doi:
10.1145/3176364.3176374.
[22] Recommended value of block size for Intel processor. Available:
https://software.intel.com/content/www/us/en/develop/documentation/m
kl-linux-developer-guide/top/intel-math-kernel-library-
benchmarks/intel-distribution-for-linpack-benchmark/configuring-
parameters.html/.
[23] Netlib. Available: https://www.netlib.org/.
[24] KISTI. Available: https://www.kisti.re.kr/.