ona Parallel Processing... Parallel applications led to parallel computing Coarse-grained systems — SMP (Symmetrical Multi Processing): Medium-grained systems — Clusters; Fine-grained systems — MPP (Massively Parallel Processing); The intention behind all of these schemes is to divide up large computational tasks across large numbers of processors; Performance improvements intended to be proportional to the number of processors, but limited by Amdahl's Law; These schemes were well established by the 1990s but with the exception of SMP, usage was mostly limited to specialised applications, mainly in scientific computing. FITS174Parallel& Dietrbuted Systems Dr. Ronald Pose Lecture 2-2013 8 = MPP A lorge parellel prozessing system with « shared. nothing architecture + Consist of several hundred nodes with o high-spend intercennection netwark/switeh + Enoch node consists of a main memory & one or mare Processors + Rune a separate copy af the OS «= SMP 1 2-64 processors today Shared-everything architecture All pracessors share all the glabal ressurces available Single copy of the OS runs.on these systemsMultitasking (cont’d) Cooperative Multitasking: one program dominates the operating system but will allow another program to run while it is idle. — Windows 3.X and MAC 0S/9 and earlier Multithreading: Data and parts of programs are shared between two or more CPUs (or multiple cores of same CPU MultitasicingInterconnection Structure srmegtergt commen ieee” Sse tanne nara sie ee (CETL, Computer Grpeniaation and Areimectane Bageereaun CHET Multistage Interconnection Network S271 Conner Oeeeeaion and ARES > aT3 Stages Memories cPUs 3}/8] (8]|5|/8} SilE IIE i 8] [8 a] [a] [8 =| [2] Poo fF] (Fl f=) () fF) fF) OTF | Sy EI TE el 3) [F z fe) TS) fF 2 ls a) |e = =) TF) fF) fF 2! te} 18! lel lel ls} lecame] (eaciz] [eacts] [acnecpu | feru] [eru ] Distributed Memory Multiprocessor a Memory Mult iprocessar Shared vs DistributedFIG. 1. A TOKEN-RING STRUCTURE 19 P] TF] le ¥ it P P| dain = dna ut alock —| singe A | stage B | sageC \ { ' ; a a a Q SeriaLin, pavalel-out shiltregister with 4-stages