TS.

Ng Vn Thanh,
Vin Vt l.

Chuyn ngnh : Cng ngh thng tin.

Michael J. Quinn. Parallel Computing, theory and practice, 2nd edition. Oregon State University, USA. McGraw Hill Inc.
Dimitri P. Bertsekas and John N. Tsitsiklis. Parallel and Distributed Computation, Numerical Methods. Massachusets Institute of Technology. Prentice Hall Press. Enrico P. The lecture notes on Parallel Computing (CS 491). Computer Science Dept. New Mexico State University, USA. Spring 2004. Thomas H. Cormen, Charles E. Leiserson, Ronald L. Rivest and Clifford Strein. Introduction to Algorithms, 2nd edition. Massachusets Institute of Technology. The MIT Press. David A. Patterson and John L. Hennessy. Computer Organization & Design, the hardware/software interface, 2nd edition. Stanford University & University of California at Berkeley, 2002. Charles Leiserson. The lecture notes on Theory of Parallel Systems. Massachusets Institute of Technology, Open Course Ware. www.ocw.mit.edu Michel Cosnard & Denis Trystram, Parallel Algorithms and Architectures. International Thomson Computer Press. 1995.

@2009, Ng Vn Thanh - Vin Vt L

Chng 1: Cc kin trc song song 1.1 Tng quan v tnh ton song song 1.2 Cc kin trc song song ng b

1.3 Cc kin trc song song khng ng b

1.2.1 My tnh mt dng lnh, mt dng d liu (SISD) 1.2.2 My tnh mt dng lnh, nhiu dng d liu (SIMD) 1.2.3 My tnh vector. 1.3.1 My tnh nhiu dng lnh, mt dng d liu (MISD hay Reduction). 1.3.2 My tnh nhiu dng lnh, nhiu dng d liu (MIMD)

1.4 T chc cc b vi x l 1.5 Cc mng b vi x l (processor arrays). 1.6 My tnh nhiu b vi x l (multiprocessor)

1.7 H thng nhiu my tnh (Multicomputers). 1.8 Gii thiu v my tnh CM-200 (Connection Machine 200 by Thinking Machine Corp).
@2009, Ng Vn Thanh - Vin Vt L

1.6.1 Nhiu b vi x l truy cp b nh ng b (Uniform Memory Access Multiprocessors). 1.6.2 Nhiu b vi x l truy cp b nh khng ng b (Non Uniform Memory Access)

Chng 1: Cc kin trc song song

1.1 Tng quan v tnh ton song song
Hn ch ca my tnh

Tc ca b vi x l ca my tnh (processor). Dung lng ca b nh RAM. Gii quyt nhng bi ton ln mt nhiu thi gian. Phn cng: Siu my tnh: SIMD, MIMD my tnh c nhiu b vi x l (multi-processor), c nhiu li trong mt b vi x l (multi-core).
Cm my tnh (cluster): kt ni cc my tnh PC vi nhau to thnh cm

Tnh ton song song

my tnh, cc my tnh giao tip vi nhau thng qua mng.

My tnh o song song (PVM = parallel vitual machine): Chia mt CPU

thnh nhiu CPU o m n c kh nng hot ng tng t nh cc CPU thc th.

@2009, Ng Vn Thanh - Vin Vt L

Phn mm lp trnh (C/C++ v Fortran):

OpenMP : ch hot ng trn h my tnh chia s b nh (shared memory),

c kin trc kiu SIMD.

MPI (message passing interface): thc hin cc cu lnh song song trn cc

CPU khc nhau bng giao thc gi v nhn cc thng ip.

High performance (HPF/HPC): tnh ton song song gia cc phn t ca cc

mng d liu. Hay ni cch khc, cc mng d liu c phn b trn cc CPU khc nhau (Data distribution). MPI v HPF/HPC u c th s dng trn c my tnh multi-processor (SIMD /MIMD), cm my tnh v c trn my tnh o song song.

Thut ton song song

Thut ton song song nhn hai ma trn. Thut ton sp xp. Thut ton song song tm kim t hp.

@2009, Ng Vn Thanh - Vin Vt L

1.2 Cc kin trc song song ng b

1.2.1 My tnh mt dng lnh, mt dng d liu (SISD - Single Instruction - Single Data)

trong mi chu trnh. V d v php tnh cng hai s a v b: cn phi thc hin 5 chu trnh.
Nhn ch th c gi tr a c gi tr b Tnh a+b Ghi kt qu

My tnh tun t (scalar computer): ch thc hin mt ch th (mt dng lnh)

Trong thc t th mi mt chu trnh c kt hp bi nhiu chu trnh con. My tnh loi SISD l my tnh PC thng thng, ch c duy nht mt b vi

x l. Khng c cu trc song song v cng khng c d liu song song.

@2009, Ng Vn Thanh - Vin Vt L

Knh tuyn (pipeline): chia cc bc tnh khc nhau thnh cc n v chc nng a ra kt qu trung gian. B vi x l c to bi nhiu knh tuyn c gi l my tnh siu tun t (superscalar computer)
Ghi kt qu Tnh a+b c gi tr b c gi tr a c gi tr a c gi tr b c gi tr a Tnh a+b c gi tr b c gi tr a

Chu trnh

c lnh

c lnh

c lnh

c lnh

c lnh

Thi gian

@2009, Ng Vn Thanh - Vin Vt L

K thut knh tuyn: V d v 5 php tnh cng hai s ai v bi: cn phi thc hin 9 chu trnh.
CT1 Lnh 1 c lnh CT2 c s a1 c lnh CT3 c s b1 c s a2 c lnh CT4 Tnh a1+b1 c s b2 c s a3 c lnh CT5 Ghi kt qu Tnh a2+b2 c s b3 c s a4 c lnh Ghi kt qu Tnh a3+b3 c s b4 c s a5 Ghi kt qu Tnh a4+b4 c s b5 Ghi kt qu Tnh a5+b5 Ghi kt qu CT6 CT7 CT8 CT9

Lnh 2 Lnh 3 Lnh 4 Lnh 5

u im: tng tc x l d liu Nhc im: Ch hot ng tt cho mt s kiu d liu c th Cu trc b vi x l phc tp Kh iu khin.
@2009, Ng Vn Thanh - Vin Vt L

1.2.2 My tnh mt dng lnh, nhiu dng d liu (SIMD - Single Instruction - Multiple Data)

i vi cc b d liu ln, cc tnh ton s thng c thc hin vi cng mt php tnh vi cc d liu khc nhau. SIMD: c th cng thc hin cng mt dng lnh trn nhiu phn d liu khc nhau. Cc bi ton s dng trn h SIMD cn phi c vector ha. My tnh vector c s dng ht sc rng ri trong cc lnh vc tnh ton hiu nng cao. Tuy nhin , cc my tnh vector c cu trc rt phc tp v gi thnh cao.

@2009, Ng Vn Thanh - Vin Vt L

SIMD own local memory

@2009, Ng Vn Thanh - Vin Vt L

Cc b vi x l truyn d liu cho nhau qua module nh

@2009, Ng Vn Thanh - Vin Vt L

1.2.3 My tnh vector

Mt my tnh vector bao gm mt b tnh ton v hng bnh thng dng k thut knh tuyn v mt b lm tnh vector .
B tnh ton v hng, ging nh b x l dng k thut ng dn, thc hin cc php tnh v hng, cn b lm tnh vector thc hin cc php tnh vector. a s cc my tnh vector cho php lm cc php tnh trn vector s nguyn, vector s l v vector s logic (s Boolean). C 2 kiu kin trc my tnh vector :

Kiu vector nh - nh: cc php tnh vector c thc hin trong b nh

Kiu thanh ghi: my tnh c nhiu thanh ghi, mi thanh ghi c nhiu vector My CRAY-2 (1995) c 8 thanh ghi vector, mi thanh ghi c th cha 64 vector , mi vector c chiu di 64 bit.

@2009, Ng Vn Thanh - Vin Vt L

1.3 Cc kin trc song song khng ng b

1.3.1 My tnh nhiu dng lnh, mt dng d liu (MISD - Multiple Instruction - Single Data)

B vi x l thc hin nhiu cu lnh khc nhau trn cng mt dng d liu. Loi my tnh ny khng ph bin, n ch c thit k ch to cho nhng nhu cu c bit.

@2009, Ng Vn Thanh - Vin Vt L

1.3.2 My tnh nhiu dng lnh, nhiu dng d liu (MIMD - Multiple Instruction - Multiple Data)

Cc my tnh c t 2 CPU hoc nhiu hn 2 li trn mt CPU c gi l MIMD. Cc b vi x l thc hin nhiu cu lnh khc nhau trn cc tp d liu khc nhau.

@2009, Ng Vn Thanh - Vin Vt L

 Shared

memory

Message

Passing

@2009, Ng Vn Thanh - Vin Vt L

1.4 T chc cc b vi x l

B x l c chia ch yu thnh hai b phn: Phn iu khin v phn ng i ca d liu.

ng i ca d liu

Cc b phn: B phn lm tnh v lun l (ALU: Arithmetic and Logic Unit). Cc mch dch. Cc thanh ghi: thanh ghi trng thi, thanh ghi chng trnh Cc ng kt ni cc b phn trn vi nhau. B a hp: ni cc knh d liu vi CPU v b nh. Nhim v: c cc ton hng t cc thanh ghi tng qut Thc hin cc php tnh trn ton hng ny trong b lm tnh v lun l ALU Lu tr kt qu trong cc thanh ghi tng qut.

@2009, Ng Vn Thanh - Vin Vt L

Phn iu khin:

B iu khin to cc tn hiu iu khin di chuyn s liu, iu khin cc tc v nh c v ghi d liu ln b nh. B iu khin mch in t: n gin v hu hiu khi cc lnh c chiu di c nh, c dng thc n gin.

@2009, Ng Vn Thanh - Vin Vt L

B iu khin vi chng trnh: iu khin cc tc v ca mt lnh m my c thc hin bng mt chui cc vi lnh. Mi mt vi chng trnh c x l bi mt vi my tnh nm bn trong b iu khin. B x l ny c th x l cc tp lnh phc tp m cc lnh c chiu di khc nhau.

@2009, Ng Vn Thanh - Vin Vt L

1.5 Cc mng b vi x l (processor arrays).

Thuc kiu kin trc SIMD. Cc b vi x l c phn b di dng mng mt chiu hoc hai chiu. Mi mt b vi x l c truy cp vi b nh ni b. Ch giao tip d liu vi cc b vi x l ln cn thng qua kt ni mng. Tt c cc b vi x l c th thc hin ng thi mt cu lnh.

@2009, Ng Vn Thanh - Vin Vt L

1.6 My tnh nhiu b vi x l (multiprocessor).

My tnh nhiu b vi x l c t nht l 2 b vi x l c kt ni vi nhau c th thc hin ng thi cc cng vic khc nhau. Phn loi theo kiu giao tip: Nhiu b vi x l (2-256) Nhiu my tnh (8-256) Phn loi theo kiu kt ni vt l: Kt ni trn Bus (2-32) Kt ni qua mng (8-256) Phn loi kiu Flynn: phn loi theo kh nng x l cc cu lnh v d liu My tnh mt dng lnh, mt dng d liu (SISD) My tnh mt dng lnh, nhiu dng d liu (SIMD) My tnh nhiu dng lnh, mt dng d liu (MISD) My tnh nhiu dng lnh, nhiu dng d liu (MIMD)
@2009, Ng Vn Thanh - Vin Vt L

Kt ni trn mt bus

@2009, Ng Vn Thanh - Vin Vt L

Kt ni y trn nhiu bus

@2009, Ng Vn Thanh - Vin Vt L

Kt ni n trn nhiu bus

@2009, Ng Vn Thanh - Vin Vt L

Kt ni tng phn trn nhiu bus

@2009, Ng Vn Thanh - Vin Vt L

Kt ni phn lp

@2009, Ng Vn Thanh - Vin Vt L

1.6.1. Nhiu b vi x l truy cp b nh ng b (Uniform Memory Access Multiprocessors).

My tnh c kin trc kiu MIMD vi mt b nh chia s (shared memory). Bao gm cc b vi x l c lp vi nhau. Tt c cc b vi x l u c th truy cp n b nh chung, thi gian truy cp n b nh ca cc b vi x l l nh nhau. Mi mt b vi x l c th c cc thanh ghi, b m, b nh ring ca mnh. Cc b vi x l kt ni vi nhau thng qua kiu kt ni mng (interconnection network), kt ni kiu mt Bus kt ni kiu nhiu Bus Kt ni kiu thanh ngang Kt ni kiu b nh a cng Nhc im: gp kh khn trong vic nng cp b nh khi gii quyt cc bi ton ln.
@2009, Ng Vn Thanh - Vin Vt L

1.6.2 Nhiu b vi x l truy cp b nh khng ng b (Non Uniform Memory Access Multiprocessors).

Tng t nh kin trc nhiu b vi x l truy cp b nh ng b. Mi mt b vi x l c gn vi mt b nh ring. u im: tng tc truy cp vao b nh. Nhc im: thi gian giao tip gia cc b vi x l ph thuc vo tc truyn qua mng

@2009, Ng Vn Thanh - Vin Vt L

1.7 H thng nhiu my tnh (Multicomputers)

H thng cluster: cc my tnh c lp c kt ni vi nhau thng qua mt h thng kt ni tc cao. Mi my tnh trong h cluster c gi l mt node. Cc my tnh c lp c th l cc my PC thng thng c mt hoc nhiu b vi x l. H thng kt ni cc my tnh vi nhau dng mng LAN bng cp thng hoc cp quang. H thng cluster c cc u im: Tc cao: Vi s h tr ca cc phn mm tnh ton song song, cluster c kh nng x l mnh hn bt c mt my tnh n l no. D nng cp: c th nng cp cho tng node hoc thm cc node. tin cy cao: H thng vn hot ng n nh khi c mt nt (node) trong h thng b h hng. Gi thnh thp: v cluster c kt ni bi cc my tnh PC thng thng.
@2009, Ng Vn Thanh - Vin Vt L

1.7 H thng nhiu my tnh (Multicomputers)

H thng cluster: cc my tnh c lp c kt ni vi nhau thng qua mt h thng kt ni tc cao. Mi my tnh trong h cluster c gi l mt node. Cc my tnh c lp c th l cc my PC thng thng c mt hoc nhiu b vi x l. H thng kt ni cc my tnh vi nhau dng mng LAN bng cp thng hoc cp quang. H thng cluster c cc u im: Tc cao: Vi s h tr ca cc phn mm tnh ton song song, cluster c kh nng x l mnh hn bt c mt my tnh n l no. D nng cp: c th nng cp cho tng node hoc thm cc node. tin cy cao: H thng vn hot ng n nh khi c mt nt (node) trong h thng b h hng. Gi thnh thp: v cluster c kt ni bi cc my tnh PC thng thng.
@2009, Ng Vn Thanh - Vin Vt L

1.8 Gii thiu v my tnh CM-200 (Connection Machine 200 by Thinking Machine Corp).

Ra i vo nm 1987, kt thc vo nm 1991. Thuc kiu kin trc SIMD. S processor: 2048 65536 Tc l thuyt: 31 Gflop/s

@2009, Ng Vn Thanh - Vin Vt L

@2009, Ng Vn Thanh - Vin Vt L

@2009, Ng Vn Thanh - Vin Vt L