P. 1
Parallel Computing

Parallel Computing

Ratings: (0)|Views: 3,349|Likes:
Published by rdorbala

More info:

Published by: rdorbala on Aug 09, 2010
Copyright:Attribution Non-commercial


Read on Scribd mobile: iPhone, iPad and Android.
download as PDF, TXT or read online from Scribd
See more
See less





Table of Contents
1.Abstract 2.Overview 1.What is Parallel Computing? 2.Why Use Parallel Computing? 3.Concepts and Terminology 1.von Neumann Computer Architecture 2.Flynn's Classical Taxonomy 3.Some General Parallel Terminology 4.Parallel Computer Memory Architectures 1.Shared Memory 2.Distributed Memory 3.Hybrid Distributed-Shared Memory 5.Parallel Programming Models 1.Overview 2.Shared Memory Model 3.Threads Model 4.Message Passing Model 5.Data Parallel Model 6.Other Models 6.Designing Parallel Programs 1.Automatic vs. Manual Parallelization 2.Understand the Problem and the Program 3.Partitioning 4.Communications 5.Synchronization 6.Data Dependencies 7.Load Balancing 8.Granularity 9.I/O 10.Limits and Costs of Parallel Programming 11.Performance Analysis and Tuning 7.Parallel Examples 1.Array Processing 2.PI Calculation 3.Simple Heat Equation 4.1-D Wave Equation 8.References and More Information 
Introduction to Parallel Computing
LC Workshops
Privacy & Legal Notice
 Blaise Barney, Lawrence Livermore National Laboratory
Page 1 of 58Introduction to Parallel Computing7/30/2010https://computing.llnl.gov/tutorials/parallel_comp/ 
 This tutorial covers the very basics of parallel computing, and is intended for someone who is justbecoming acquainted with the subject. It begins with a brief overview, including concepts andterminology associated with parallel computing. The topics of parallel memory architectures andprogramming models are then explored. These topics are followed by a discussion on a number of issuesrelated to designing parallel programs. The tutorial concludes with several examples of how toparallelize simple serial programs.Level/Prerequisites: None
What is Parallel Computing?
Traditionally, software has been written for
To be run on a single computer having a single Central Processing Unit (CPU);
A problem is broken into a discrete series of instructions.
Instructions are executed one after another.
Only one instruction may execute at any moment in time.
In the simplest sense,
 parallel computing
is the simultaneous use of multiple compute resourcesto solve a computational problem:
To be run using multiple CPUs
A problem is broken into discrete parts that can be solved concurrently
Each part is further broken down to a series of instructions
Instructions from each part execute simultaneously on different CPUs
Page 2 of 58Introduction to Parallel Computing7/30/2010https://computing.llnl.gov/tutorials/parallel_comp/ 
The compute resources can include:
A single computer with multiple processors;
An arbitrary number of computers connected by a network;
A combination of both.
The computational problem usually demonstrates characteristics such as the ability to be:
Broken apart into discrete pieces of work that can be solved simultaneously;
Execute multiple program instructions at any moment in time;
Solved in less time with multiple compute resources than with a single compute resource.
The Universe is Parallel:
Parallel computing is an evolution of serial computing that attempts to emulate what has alwaysbeen the state of affairs in the natural world: many complex, interrelated events happening at thesame time, yet within a sequence. For example:
Galaxy formation
Planetary movement
Weather and ocean patterns
Tectonic plate drift
Rush hour traffic
Automobile assembly line
Building a space shuttle
Ordering a hamburger at the drive throu
The Real World is Massively Parallel
Page 3 of 58Introduction to Parallel Computing7/30/2010https://computing.llnl.gov/tutorials/parallel_comp/ 

Activity (32)

You've already reviewed this. Edit your review.
1 thousand reads
1 hundred reads
rajaramansuresh liked this
akbisoi1 liked this
Kunal Kaul liked this
nishinv liked this
Rajes.G liked this
Chennappa Sugali liked this
SAPATH liked this

You're Reading a Free Preview

/*********** DO NOT ALTER ANYTHING BELOW THIS LINE ! ************/ var s_code=s.t();if(s_code)document.write(s_code)//-->