PARTNER SHOWCASE
NVIDIA is a MSC Software Performance partner with Quadro® and Professional Solution product lines that provide excellent
performance for Patran and MSC Nastran on Windows® and Linux® systems.
MSC Software: Partner Showcase - NVIDIA
GPU Computing Accelerates
Simulation Performance for MSC Nastran Users
The power wall (resulting from increase in
power consumption and heat dissipation due
to increased processor speeds) has introduced
radical changes in computer architectures.
Increasing core counts and hence, increasing
parallelism have replaced increasing clock
speeds as the primary way of delivering greater
hardware performance. A modern GPU (Graphics
Processing Unit) consists of hundreds of simple
processing cores; this degree of parallelism
on a single processor is typically referred to as
‘many-core’ relative to ‘multi-core’ that refers to
processors with at most a few dozen cores.
Many-core GPUs will often demand a high
degree of fine-grained parallelism – the
application program should create many threads
so that while some threads are waiting for data
to return from memory other threads can be
executing – offering a different approach in
Key Highlights:
terms of hiding memory latency because of their
specialization to inherently parallel problems.
With the ever-increasing demand for more Industry
computing performance, the HPC industry is High-Perfomance
moving towards a hybrid computing model, Computing
where GPUs and CPUs work together to
perform general purpose computing tasks. In
this hybrid-computing model, the GPU serves Challenge
as a co-processor to the CPU. Co-processing Increase computing performance by
refers to the use of an accelerator, a GPU, to developing a hybrid computing model
offload the CPU and to increase computational
efficiency. In order to exploit this hybrid MSC Software Solutions
computing model and the massively parallel
MSC Nastran 2012 to support GPU
GPU architecture, application software will need
computing capability including multiple GPU
to be redesigned. MSC Software and NVIDIA
computing capability for DMP runs
engineers have been working together over
the last year on the use of GPUs to accelerate
the sparse direct solver in MSC Nastran. Benefits
• Vastly reduce use of pinned host memory
• Handle arbitrarily large fronts, for very
large models
Partner Showcase: NVIDIA | 1
 PARTNER SHOWCASE
Solver Acceleration in
MSC Nastran 2012:
A sparse direct solver is possibly the most
important component in a finite element
structural analysis program. Typically, a
multi-frontal algorithm with out-of-core
capability for solving extremely large
problems and BLAS level 3 kernels for the
highest compute efficiency is implemented.
Elimination tree and compute kernel level
parallelism with dynamic scheduling is used
to ensure the best scalability. The BLAS level
3 compute kernels in a sparse direct solver
are the prime candidate for GPU computing
due to their high floating point density and
favorable compute to communication ratio.
The proprietary symmetric MSCLDL and
asymmetric MSCLU sparse direct solvers
in MSC Nastran employ a super-element
analysis concept instead of dynamic tree level
parallelism. In this super-element analysis, the
structure/matrix is first decomposed into large
sub-structures/sub-domains according to user
input and load balance heuristics. The out-
of-core multi-frontal algorithm is then used to
compute the boundary stiffness, or the Schur
compliment, followed by the transformation of
the load vector, or the right hand side, to the
boundary. The global solution is found after
the boundary stiffness matrices are assembled
into the residual structure and the residual
structure is factorized and solved. The GPU is
a natural fit for each sub-structure boundary
stiffness/Schur compliment calculation.
Today’s GPUs can provide memory bandwidth
Figure 1: NVIDIA Tesla 20-series GPUs (workstation & server form factors)
2 | MSC Software
and floating-point performance that are several
factors faster than the latest CPUs. In
MSC Nastran, the most time consuming part is
the BLAS level 3 operations in the multi-frontal
factorization process. To date, only the trailing
matrix updates of the front factorization are
implemented as CUDA kernels and these
update kernels are the subject of a collaborative
work between NVIDIA and MSC engineers.
GPU Computing Implementation and
Target Analysis (Solution Sequences):
NVIDIA’s CUDA parallel programming
architecture is used to implement the update
kernels. CUDA is the hardware and software
architecture that enables NVIDIA GPUs
to execute programs written with C, C++,
FORTRAN, OpenCL, and other languages.
Vastly reduced use of pinned host memory
and the ability to handle arbitrarily large
fronts, for very large models (greater than
15M DOF) on a single Tesla C2050 GPU, are
some strengths of the GPU implementation
in MSC Nastran 2012. ‘Staging’ is a term that
is used to describe how very large fronts are
handled. If the trailing submatrix is too large
to fit on the GPU device memory, then it is
broken up into approximately equal-sized
‘stages’ and the stages are completed in
order. Multiple streams are used within a
stage. So, for an arbitrarily large submatrix,
say 40GB, then it would be solved in, say, 10
stages of 4GB each. The actual sizes of the
stages can be varied for performance tuning.
MSC Software: Partner Showcase - NVIDIA
In addition, the MSC Nastran implementation
supports multiple GPU computing capability
for DMP (Distributed Memory Parallel) runs.
In such cases of DMP>1, multiple fronts are
factorized concurrently on multiple GPUs. The
matrix is decomposed into two domains, and
each domain is computed by a MPI process.
A typical MSC Nastran job submission
command with multiple GPUs is shown below:
nastran2012 jid=myinput mem=48gb
buffsize=65537 dmp=2 gpuid=0:1
gputhresh=12000 sys205=192
sys151=1 mode=i8 sdir=/local/
skodiyal/tmp bat=no scr=yes
gpuid is the ID of a licensed GPU device to
be used in the analysis. Multiple IDs may
be assigned to MSC Nastran DMP runs.
gputhresh represents the minimum threshold
for GPU computing in the multi-frontal sparse
factorization. If the product of the rank size
and the front size of each front is smaller than
value, the rank update of the front is processed
on the CPU. Otherwise, the GPU device would
be used for the rank update of the front.
The GPUs supported with this implementation
are the NVIDIA Tesla 20-series (shown in
Figure 1) and Quadro GPUs based on the Fermi
architecture (compute capability 2.0). Linux
and Windows 64-bit platforms are supported
Any ‘fat’ BLAS3 code path would be
potential candidate for GPU computing.
Sparse direct solver intensive SOL101 (linear
statics), SOL108 (direct frequency) and
SOL400 (nonlinear) fall into this category.
MSC Software: Partner Showcase - NVIDIA
Figure 2: Automotive crank shaft (945K DOF) and engine (15.2M DOF) models
SOL108 would need a complex sparse direct
solver that is not supported in MSC Nastran
2012 implementation, however, this feature
is currently under development and testing
for an upcoming point release. Likewise,
conventional SOL111 (modal frequency) with
large MPYAD’s (multiply-add) also should
benefit from GPU computing in a later release.
Performance analysis with
GPU Computing:
Linear and nonlinear structural stress analysis
are the target applications with this first
implementation of GPU computing in
MSC Nastran 2012. Structural finite element
models dominated by solid elements provide
for more concentrated computational work
in the sparse matrix factorization, which
is highly desirable for the GPU. A range of
models with varying fidelity, from around
1M degrees of freedom (DOF) to 15M DOF
is considered (Figure 2). Performance
comparisons are relative to a serial Nastran
run, which is still widely adopted within
the customer community, as well as with
multi-core (2x quad-core Nehalem) CPUs.
PARTNER SHOWCASE
Figure 3: Performance speed-ups with
Single and Multiple GPUs using MSC Nastran
2012 models
The hardware configurations used with enabled by GPU computing will facilitate
these benchmark runs consisted of: MSC Nastran users to add more realism
(1) AMAX server, Linux, 2x hex-core Westmere, to their models thus improving the quality
2.67GHz, 32GB memory, 2x Tesla C2050 of the simulations. A rapid CAE simulation
GPU for the 945K and 1.3M DOF model capability from GPUs has the potential to
transform current practices in engineering
(2) Super Micro server, Linux, 2x quad- analysis and design optimization procedures.
core Nehalem 2.27GHz, 96GB memory,
2.2 TB SATA 5-way striped RAID and 2x This initial GPU computing implementation
Tesla C2050 GPU for all other models. also identified certain issues – for one, the
larger the model, the higher the DMP overhead
Figure 3 shows the end-to-end (total) speed-up in MSC Nastran. This increased CPU side
for single and multiple GPU runs. In general, overhead reduces the overall speed-up
based on the benchmark models, we see
resulting from GPU computing. Future
speed-ups in the range of 4-6X with a single releases of MSC Nastran will address such
GPU over a serial run and in the range of
issues as well as expand the GPU computing
1.4-2X with 2 GPUs over a 8 core DMP run. capability to include complex solver kernels
for the NVH and dynamics markets.
Summary:
GPU computing is implemented in
MSC Nastran 2012 to significantly lower the
simulation times for industry standard analysis
models. Vastly reduced use of pinned memory
and the ability to handle arbitrarily large front
sizes for very large models are some of the
strengths of this implementation. Further,
multiple GPUs can be used with Nastran
DMP analysis. The performance speed-ups
Partner Showcase: NVIDIA | 3
 PARTNER SHOWCASE

About MSC Software About MSC Nastran

MSC Software is one of the ten original software companies and
the worldwide leader in multidiscipline simulation. As a trusted
partner, MSC Software helps companies improve quality, save time
and reduce costs associated with design and test of manufactured
products. Academic institutions, researchers, and students employ
MSC technology to expand individual knowledge as well as expand
the horizon of simulation. MSC Software employs 1,000 professionals
in 20 countries. For additional information about MSC Software’s
products and services, please visit www.mscsoftware.com.
MSC Nastran Structural & Multidiscipline FEA
MSC Nastran is the world’s most widely used Finite Element Analysis
(FEA) solver that helped MSC Software become recognized in 2011
as one of the “10 Original Software Companies”. When it comes to
solving for stress/strain behavior, dynamic and vibration response
and thermal gradients in real-world systems, MSC Nastran is
recognized as the most trusted multidiscipline solver in the world.
MSC Nastran is built on work done by NASA scientists and
researchers, and is trusted for the design of mission critical systems
in every industry. Nearly every spacecraft, aircraft, and vehicle
designed in the last 40 years has been analyzed using MSC Nastran.
In recent years, several extensions to its capabilities have
resulted in a single multidisciplinary solver providing users
with a trusted solution to simulate everything from a single
component to complex assemblies under diverse conditions.

Please visit MSC Nastran offers a complete set of linear static and dynamic
analysis capabilities along with unparalleled support for

www.mscsoftware.com superelements enabling users to solve large, complex assemblies

more efficiently. MSC Nastran also offers a complete set of implicit

for more partner showcases and explicit nonlinear analysis capabilities, thermal and interior/
exterior acoustics, and coupling between various disciplines
such as thermal, structural, and fluid interaction. New modular
packaging that enables you to get only what you need makes
it more affordable to own MSC Nastran than ever before.

Corporate Europe, Middle East, Asia-Pacific Asia-Pacific

MSC Software Corporation
2 MacArthur Place
Santa Ana, California 92707
Telephone 714.540.8900
www.mscsoftware.com
Africa MSC Software Japan LTD. MSC Software (S) Pte. Ltd.
MSC Software GmbH Shinjuku First West 8F 100 Beach Road The MSC Software corporate logo, MSC, and the names of the
Am Moosfeld 13 23-7 Nishi Shinjuku #16-05 Shaw Tower MSC ‌Software products and services referenced herein are trademarks
81829 Munich, Germany 1-Chome, Shinjuku-Ku Singapore 189702 or registered trademarks of the MSC Software Corporation in the United
Telephone 49.89.431.98.70 Tokyo, Japan 160-0023 Telephone 65.6272.0082 States and/or other countries. All other trademarks belong to their
Telephone 81.3.6911.1200 respective owners. © 2012 MSC Software Corporation. All rights reserved.

NVIDIA*2012MAY*PS