HIGH PERFORMANCE

COMPUTING
LECTURE 1

Sabah Sayed
Course Materials
 Course: High Performance Computing
 Course Cod: CS371

 Acadox:
 http://www.acadox.com/class/64485

 Invite Code : 2KAPYE

http://www.acadox.com/join/2KAPYE

2/22/2023
2
In this Course
 An introduction to parallel programming
 Parallel Hardware and Parallel Software

 Parallel Programming development by the most

widely used application programming
interfaces (APIs).
➢ MPI
➢ OpenMP

3
 Textbook

 An Introduction to Parallel Programming,

Peter S. Pacheco

 Programming language
C on Linux (VMware)

4
Course Assessment

Assignments 18

Lab Participation 7

Midterm 15

Final 60

5
Parallel Programming Languages

6
 Why Parallel Computing
 Why we need ever-increasing performance.
 Why we’re building parallel systems.
 Why we need to write parallel programs.
 How do we write parallel programs.
 What we’ll be doing.
 Concurrent, parallel, distributed.

7
 Changing times
 From 1986 – 2002, microprocessors were speeding
like a rocket, increasing in performance an average
of 50% per year.

 Since then, it’s dropped to about 20% increase per

year.

8
An intelligent solution
 Instead of designing and building faster
microprocessors, put multiple processors on a single
integrated circuit.

9
Why we need ever-increasing
performance
 Computational power is increasing, but so are our
computation problems and needs.
 Problems such as:
 decoding the human genome.
 fast and accurate Web searches.

 more realistic computer games.

 More complex problems are still waiting to be

solved.

10
 Climate modeling

 In order to better understand climate

change, we need far more accurate
computer models.
 Interactions between the atmosphere,
the oceans, solid land, and the ice caps
at the poles.
 Make detailed studies of how various
interventions might affect the global
climate.

11
 Study complex molecules

 Our ability to study configurations of complex

molecules such as proteins is severely limited by our
current computational power.

12
 Drug discovery
 There are many ways in which
increased computational power can
be used in research into new medical
treatments.
 Drugs that are effective in treating a
small number of patients.
 Analysis of the genomes of the other
patients will involve extensive
computational analysis of genomes.

13
 Energy research

 Program much more detailed models of technologies

such as wind turbines, solar cells, and batteries.
 These programs may provide the information needed
to construct far more efficient clean energy sources.

14
 Data analysis
 The quantity of data stored worldwide doubles every
two years.
 But it is largely useless unless it’s analyzed.

15
Why we’re building parallel systems

 Up to now, performance increases have been

attributable to increasing density of transistors.

 But there are

inherent
problems.

16
Why we’re building parallel systems

 Smaller transistors = faster processors.

 Faster processors = increased power consumption.
 Increased power consumption = increased heat.
 Increased heat = unreliable processors.

17
Solution
 Move away from single-core systems to multicore
processors.
 “core” = central processing unit (CPU)

◼ Introducing parallelism!!!

18
Now it’s up to the programmers
 Adding more processors doesn’t help much if
programmers aren’t aware of them…
 … or don’t know how to use them.

 Serial programs don’t benefit from this approach (in

most cases).

19
Why we need to write parallel
programs
 Running multiple instances of a serial program often
isn’t very useful.
 Think of running multiple instances of your favorite
game.

 What you really want is for

it to run faster.

20
Approaches to the serial problem
1) Rewrite serial programs so that they’re parallel.

2) Write translation programs that automatically

convert serial programs into parallel programs.
➢ This is very difficult to do.
➢ Success has been limited.

21
More problems
 Some coding constructs can be recognized by an
automatic program generator and converted to a
parallel construct.
 However, it’s likely that the result will be a very
inefficient program.
 Sometimes the best parallel solution is to step back
and devise an entirely new algorithm.

22