HPC @ RZ.RWTH-AACHEN.

DE
Sun
Sun Fire
Fire T2000
T2000

SunFire E25K
CPU and IO boards

SunFire E25K
Cluster
FC-AL Switch
FC
FC-AL

SunFire E6900
Cluster

SunHPC 2007 - Introduction

C
C
C

SunFire V40z Cluster

Cen te r for
Comp uting an d C omm un ic ation

Agenda - Overview

9:00 10:30

Monday
March 12

Tuesday
March 13

Serial
Performance

Shared Mem
Parallelization

12:30 14:00
14:00 15:30
15:30 16:00
16:00 17:30
19:00

Tools

Tuesday
March 15

Friday
March 16

Tuning Workshop

coffee break

10:30 11:00
11:00 12:30

Wednesday
March 14

Serial
Performance

Shared Mem
Parallelization

Tools

Tuning Workshop

lunch (Mensa Vita) - 20 min. walk or bus shuttle

Shared Mem Parallelization

Lab /Tuning
Workshop

Tuning
Workshop

coffee break
Shared Mem Parallelization

Lab /Tuning
Workshop

Social dinner
rest. "Kazan"

SunHPC 2007 - Introduction

Tuning
Workshop

C
C
C

Cen te r for
Comp uting an d C omm un ic ation

Compute Cluster RWTH Aachen

Compute Cluster der RWTH Aachen
Feb 07

#nodes

model

processor
type

SF E25K

UltraSPARC IV

72

144

144

1050

288

16
8
1

SF E6900
SF E2900
SF T2000

UltraSPARC IV
UltraSPARC IV
UltraSPARC T1

24
12
1

48
24
8

48
24
32

1200
1200
1000

96
48
8

64

SF V40z

Opteron 848

2200

SF V40z

Opteron 875

2200

16

SF X4600

16

16

2600

32

9
108

Dell 1950
sum

Opteron 885
Xeon 5160
(Woodcrest)

2
1042

4
1833

4
1857

3000

clock memory
#procs #cores #threads [MHz] [GB]

network
Gigabit Ethernet
Fire Link
Gigabit Ethernet
Fire Link
Gigabit Ethernet
Gigabit Ethernet

accumulated accumulated
performance memory
[TFLOPS]
[TB]
1.21

1.15

1.84
0.46
0.0001

1.54
0.38
0.01

Gigabit Ethernet
Gigabit Ethernet
Infiniband

1.13

0.51

0.14

0.06

Gigabit Ethernet
Gigabit Ethernet
Infiniband

0.17

0.06

0.22
5.16

0.07
3.79

SunHPC 2007 - Introduction

C
C
C

Cen te r for
Comp uting an d C omm un ic ation

Overview over HPC Tools

Current program development environment for HPC on

the Sun UltraSPARC and Opteron/Xeon systems at the
RWTH
4 platforms :
1. SPARC/Solaris 9/10, 64bit
2. Opteron/Solaris, 64 bit
3. Opteron/Linux and Intel/Linux, 64 bit
4. Opteron/Windows and Intel/Windows, 64 bit
serial programming, shared memory parallelization, message
passing
compilers / MPI libraries, debugging tools, performance analysis
tools

SunHPC 2007 - Introduction

C
C
C

Cen te r for
Comp uting an d C omm un ic ation

Programming Environment
Compilers + Debugging Tools
Company

Compiler
Version

Language

OpenMP support

Autopar

Debugger

Runtime
Analysis

US
IV

Opteron

Xeon

Solaris Lin Win Lin Win

analyzer,
collect,
er_print,
gprof
analyzer,
collect,
er_print,
gprof

Sun

Studio 11

F95/C/C++

F95/C++

F95/C++

dbx
sunstudio

Sun

Studio 12

F95/C/C++

F95/C++

F95/C++

dbx
sunstudio

Intel

V9.1

F95/C++

F95/C++

idb

vtune

X vtune vtune

Intel

V10.0 beta

F95/C++

F95/C++

idb

vtune

X vtune vtune

Intel

Guide V4.0

F90/C/C++

GNU
GNU
PGI

V4.0
V4.2 beta
V6.1
Visual Studio
2003
Visual Studio
2005

F95/C++
F95/C++
F77/F90/C/C++

X
X
X

Microsoft
Microsoft
Etnus

F95/C++
Threading Tools
F95/C++
Threading Tools
F77/F90/C/C++
Assure
F95/C++
F77/F90/C/C++

C++
C++

C++

F77/F90/C/C++

F77/F90/C/C++

GuideView
gdb
gdb
pgdbg

gprof
gprof
pgprof

X
X
X

Visual Studio

Visual Studio

TotalView 8.0

SunHPC 2007 - Introduction

C
C
C

Cen te r for
Comp uting an d C omm un ic ation

Programming Environment
MPI Libraries and Tools
Company
Sun
Sun
Sun

Version

MPI2 support

HPC ClusterTools 5
HPC ClusterTools 6
HPC ClusterTools 7
based on OpenMPI

Debugger

Plattform

Network

yes
yes

TotalView, Prism
TotalView

analyzer, mpprof
analyzer, mpprof

Solaris 9
Solaris 10

tcp, shm, FireLink

tcp, shm

yes

TotalView

analyzer

Solaris 10

tcp, shm, infiniband

Linux

tcp, shm, Infiniband

Sol, Lin, Win

Sol, Lin, Win
Linux

tcp, shm
tcp, shm
tcp, shm, Infiniband

Windows

tcp, shm, (Infiniband)

Sol, Lin

any

Intel

based on mpich2.x

yes

TotalView

ANL
ANL
Voltaire

mpich 1.2.6
mpich2 1.0.x
based on mpich 1.2.5

no
yes
no

Microsoft

based on mpich2

yes

TotalView
TotalView
?
Visual Studio w/
MS Compute
Cluster Pack

Univ Dresden

installation planned

Runtime
Analysis

Intel
TraceCollector
& Analyzer
jumpshot

Vampir-NG

SunHPC 2007 - Introduction

C
C
C

Cen te r for
Comp uting an d C omm un ic ation

Application Software
FEM
Abaqus
Ansys
Hyperworks
Ls-Dyna
Marc/Mentat
Nastran/Patran

Chemie
Gaussian
Turbomole

CFD
CFX/TASCFlow
Fluent
ICEM-CFD
StarCD

Mathematik
Maple
Mathematica

Misc
Matlab/Simulink
Tecplot

Site license
Parallel version available
No site license

SunHPC 2007 - Introduction

C
C
C

Cen te r for
Comp uting an d C omm un ic ation

License Conditions and Fees

Commercial Application Software
the responsilitiy of each user
no commercial usage
fees covered by the institutions
Commercial development tools are provided by the
CCC
Compilers
MPI
Debuggers
Runtime Analysis tools
8

SunHPC 2007 - Introduction

C
C
C

Cen te r for
Comp uting an d C omm un ic ation

System Management

Frontend nodes for interactive work,

program development and testing,
GUIs
cluster.rz.RWTH-Aachen.DE (USIV-Solaris 9)
cluster-sol10.rz.RWTH-Aachen.DE (USIV-Solaris 10)
cluster-sun-x86.rz.RWTH-Aachen.DE (OPT-Solaris 10)
cluster-linux.rz.RWTH-Aachen.DE (Opteron-Linux 64b)
cluster-win.rz.RWTH-Aachen.DE (Intel-Windows 64b)

Batch system Sun Grid Engine: jobs (> 20 min)

SunHPC 2007 - Introduction

C
C
C

Cen te r for
Comp uting an d C omm un ic ation

Sun Grid Engine

Job submission
qsub [options] scriptfile [- scriptarguments]

Job deletion
qdel job_id

Job status inquiry

qstat [options]
-help
-f #all queues and jobs
-j job_id
-u user

Graphical Interface
qmon

10

SunHPC 2007 - Introduction

C
C
C

Cen te r for
Comp uting an d C omm un ic ation

www.rz.rwth-aachen.de/computing/hpc/statistics/load/

11

SunHPC 2007 - Introduction

C
C
C

Cen te r for
Comp uting an d C omm un ic ation

Some Web-links

Web page of the SunHPC workshop with more links and information
http://www.rz.rwth-aachen.de/sunhpc

Information related to HPC at the RWTH

http://www.rz.rwth-aachen.de/hpc/

Information related to the RWTH Sun Fire SMP Cluster

http://www.rz.rwth-aachen.de/hpc/sun/

Sun Fire SMP Cluster Primer

http://www.rz.rwth-aachen.de/hpc/primer

Further reading: Batch system

http://www.rz.rwth-aachen.de/hpc/batch

Further reading: Software

http://www.rz.rwth-aachen.de/computing/sw

12

SunHPC 2007 - Introduction

C
C
C

Cen te r for
Comp uting an d C omm un ic ation

Questions? Problems?
Just send an e-mail to our trouble ticket system:
Software and Licensing:
sw@rz.rwth-aachen.de
Batch system:
batch@rz.rwth-aachen.de
Support inquiries:
support@rz.rwth-aachen.de
Programming, Debugging, Tuning, Parallelization:
hpc@rz.rwth-aachen.de

13

SunHPC 2007 - Introduction

C
C
C

Cen te r for
Comp uting an d C omm un ic ation