TS.

Ng Vn Thanh,
Vin Vt l.

Chuyn ngnh : Cng ngh thng tin.

Michael J. Quinn. Parallel Computing, theory and practice, 2nd edition. Oregon
State University, USA. McGraw Hill Inc.

Dimitri P. Bertsekas and John N. Tsitsiklis. Parallel and Distributed Computation,

Numerical Methods. Massachusets Institute of Technology. Prentice Hall Press.

Enrico P. The lecture notes on Parallel Computing (CS 491). Computer Science
Dept. New Mexico State University, USA. Spring 2004.

Thomas H. Cormen, Charles E. Leiserson, Ronald L. Rivest and Clifford Strein.

Introduction to Algorithms, 2nd edition. Massachusets Institute of Technology.
The MIT Press.
David A. Patterson and John L. Hennessy. Computer Organization & Design, the
hardware/software interface, 2nd edition. Stanford University & University of
California at Berkeley, 2002.

Charles Leiserson. The lecture notes on Theory of Parallel Systems.

Massachusets Institute of Technology, Open Course Ware. www.ocw.mit.edu

Michel Cosnard & Denis Trystram, Parallel Algorithms and Architectures.

International Thomson Computer Press. 1995.

@2009, Ng Vn Thanh - Vin Vt L

Chng 1: Cc kin trc song song

1.1 Tng quan v tnh ton song song
1.2 Cc kin trc song song ng b

1.2.1 My tnh mt dng lnh, mt dng d liu (SISD)

1.2.2 My tnh mt dng lnh, nhiu dng d liu (SIMD)
1.2.3 My tnh vector.

1.3 Cc kin trc song song khng ng b

1.3.1 My tnh nhiu dng lnh, mt dng d liu (MISD hay Reduction).
1.3.2 My tnh nhiu dng lnh, nhiu dng d liu (MIMD)

1.4 T chc cc b vi x l
1.5 Cc mng b vi x l (processor arrays).
1.6 My tnh nhiu b vi x l (multiprocessor)

1.6.1 Nhiu b vi x l truy cp b nh ng b (Uniform Memory Access

Multiprocessors).
1.6.2 Nhiu b vi x l truy cp b nh khng ng b (Non Uniform Memory
Access)

1.7 H thng nhiu my tnh (Multicomputers).

1.8 Gii thiu v my tnh CM-200 (Connection Machine 200 by Thinking
Machine Corp).
@2009, Ng Vn Thanh - Vin Vt L

Chng 1: Cc kin trc song song

1.1 Tng quan v tnh ton song song
Hn ch ca my tnh

Tc ca b vi x l ca my tnh (processor).
Dung lng ca b nh RAM.
Gii quyt nhng bi ton ln mt nhiu thi gian.

Tnh ton song song

Phn cng:
Siu my tnh: SIMD, MIMD
my tnh c nhiu b vi x l (multi-processor), c nhiu li trong mt b
vi x l (multi-core).
Cm my tnh (cluster): kt ni cc my tnh PC vi nhau to thnh cm

my tnh, cc my tnh giao tip vi nhau thng qua mng.

My tnh o song song (PVM = parallel vitual machine): Chia mt CPU

thnh nhiu CPU o m n c kh nng hot ng tng t nh cc CPU

thc th.
@2009, Ng Vn Thanh - Vin Vt L

Phn mm lp trnh (C/C++ v Fortran):

OpenMP : ch hot ng trn h my tnh chia s b nh (shared memory),

c kin trc kiu SIMD.

MPI (message passing interface): thc hin cc cu lnh song song trn cc

CPU khc nhau bng giao thc gi v nhn cc thng ip.

High performance (HPF/HPC): tnh ton song song gia cc phn t ca cc

mng d liu. Hay ni cch khc, cc mng d liu c phn b trn cc

CPU khc nhau (Data distribution).
MPI v HPF/HPC u c th s dng trn c my tnh multi-processor
(SIMD /MIMD), cm my tnh v c trn my tnh o song song.

Thut ton song song

Thut ton song song nhn hai ma trn.
Thut ton sp xp.
Thut ton song song tm kim t hp.

@2009, Ng Vn Thanh - Vin Vt L

1.2 Cc kin trc song song ng b

1.2.1 My tnh mt dng lnh, mt dng d liu (SISD - Single
Instruction - Single Data)

My tnh tun t (scalar computer): ch thc hin mt ch th (mt dng lnh)

trong mi chu trnh.

V d v php tnh cng hai s a v b: cn phi thc hin 5 chu trnh.
Nhn
ch th

c gi
tr a

c gi
tr b

Tnh
a+b

Ghi kt
qu

Trong thc t th mi mt chu trnh c kt hp bi nhiu chu trnh con.

My tnh loi SISD l my tnh PC thng thng, ch c duy nht mt b vi

x l. Khng c cu trc song song v cng khng c d liu song song.

@2009, Ng Vn Thanh - Vin Vt L

Knh tuyn (pipeline): chia cc bc tnh khc nhau thnh cc n v chc

nng a ra kt qu trung gian.
B vi x l c to bi nhiu knh tuyn c gi l my tnh siu tun t
(superscalar computer)

Chu trnh

c
lnh

Ghi kt
qu
Tnh
a+b

Tnh
a+b

c gi
tr b

c gi
tr b

c gi
tr b

c gi
tr a

c gi
tr a

c gi
tr a

c gi
tr a

c
lnh

c
lnh

c
lnh

c
lnh

Thi gian

@2009, Ng Vn Thanh - Vin Vt L

K thut knh tuyn:

V d v 5 php tnh cng hai s ai v bi: cn phi thc hin 9 chu trnh.

Lnh 1

Lnh 2
Lnh 3
Lnh 4

CT1

CT2

CT3

CT4

CT5

c
lnh

c s
a1

c s
b1

Tnh
a1+b1

Ghi kt
qu

c
lnh

c s
a2

c s
b2

Tnh
a2+b2

Ghi kt
qu

c
lnh

c s
a3

c s
b3

Tnh
a3+b3

Ghi kt
qu

c
lnh

c s
a4

c s
b4

Tnh
a4+b4

Ghi kt
qu

c
lnh

c s
a5

c s
b5

Tnh
a5+b5

Lnh 5

CT6

CT7

CT8

CT9

Ghi kt
qu

u im: tng tc x l d liu

Nhc im:
Ch hot ng tt cho mt s kiu d liu c th
Cu trc b vi x l phc tp
Kh iu khin.
@2009, Ng Vn Thanh - Vin Vt L

1.2.2 My tnh mt dng lnh, nhiu dng d liu (SIMD - Single

Instruction - Multiple Data)

i vi cc b d liu ln, cc tnh ton s thng c thc hin vi cng

mt php tnh vi cc d liu khc nhau.
SIMD: c th cng thc hin cng mt dng lnh trn nhiu phn d liu
khc nhau.
Cc bi ton s dng trn h SIMD cn phi c vector ha.
My tnh vector c s dng ht sc rng ri trong cc lnh vc tnh ton
hiu nng cao. Tuy nhin , cc my tnh vector c cu trc rt phc tp v gi
thnh cao.

@2009, Ng Vn Thanh - Vin Vt L

SIMD own local memory

@2009, Ng Vn Thanh - Vin Vt L

Cc b vi x l truyn d liu cho nhau qua module nh

@2009, Ng Vn Thanh - Vin Vt L

1.2.3 My tnh vector

Mt my tnh vector bao gm mt b tnh ton v hng bnh thng dng
k thut knh tuyn v mt b lm tnh vector .

B tnh ton v hng, ging nh b x l dng k thut ng dn, thc hin

cc php tnh v hng, cn b lm tnh vector thc hin cc php tnh
vector.

a s cc my tnh vector cho php lm cc php tnh trn vector s nguyn,

vector s l v vector s logic (s Boolean).

C 2 kiu kin trc my tnh vector :

Kiu vector nh - nh: cc php tnh vector c thc hin trong b

nh

Kiu thanh ghi: my tnh c nhiu thanh ghi, mi thanh ghi c nhiu vector

My CRAY-2 (1995) c 8 thanh ghi vector, mi thanh ghi c th cha 64

vector , mi vector c chiu di 64 bit.

@2009, Ng Vn Thanh - Vin Vt L

1.3 Cc kin trc song song khng ng b

1.3.1 My tnh nhiu dng lnh, mt dng d liu (MISD - Multiple
Instruction - Single Data)

B vi x l thc hin nhiu cu lnh khc nhau trn cng mt dng d liu.
Loi my tnh ny khng ph bin, n ch c thit k ch to cho nhng
nhu cu c bit.

@2009, Ng Vn Thanh - Vin Vt L

1.3.2 My tnh nhiu dng lnh, nhiu dng d liu (MIMD - Multiple
Instruction - Multiple Data)

Cc my tnh c t 2 CPU hoc nhiu hn 2 li trn mt CPU c gi l

MIMD.
Cc b vi x l thc hin nhiu cu lnh khc nhau trn cc tp d liu
khc nhau.

@2009, Ng Vn Thanh - Vin Vt L

 Shared

memory

Message

Passing

@2009, Ng Vn Thanh - Vin Vt L

1.4 T chc cc b vi x l

B x l c chia ch yu thnh hai b phn: Phn iu khin v phn

ng i ca d liu.

ng i ca d liu

Cc b phn:
B phn lm tnh v lun l (ALU: Arithmetic and Logic Unit).
Cc mch dch.
Cc thanh ghi: thanh ghi trng thi, thanh ghi chng trnh
Cc ng kt ni cc b phn trn vi nhau.
B a hp: ni cc knh d liu vi CPU v b nh.
Nhim v:
c cc ton hng t cc thanh ghi tng qut
Thc hin cc php tnh trn ton hng ny trong b lm tnh v lun l
ALU
Lu tr kt qu trong cc thanh ghi tng qut.

@2009, Ng Vn Thanh - Vin Vt L

Phn iu khin:

B iu khin to cc tn hiu iu khin di chuyn s liu, iu khin cc

tc v nh c v ghi d liu ln b nh.
B iu khin mch in t: n gin v hu hiu khi cc lnh c chiu di
c nh, c dng thc n gin.

@2009, Ng Vn Thanh - Vin Vt L

B iu khin vi chng trnh: iu khin cc tc v ca mt lnh m my c thc

hin bng mt chui cc vi lnh.
Mi mt vi chng trnh c x l bi mt vi my tnh nm bn trong b iu
khin.
B x l ny c th x l cc tp lnh phc tp m cc lnh c chiu di khc
nhau.

@2009, Ng Vn Thanh - Vin Vt L

1.5 Cc mng b vi x l (processor arrays).

Thuc kiu kin trc SIMD.

Cc b vi x l c phn b di dng mng mt chiu hoc hai chiu.
Mi mt b vi x l c truy cp vi b nh ni b. Ch giao tip d liu vi cc
b vi x l ln cn thng qua kt ni mng.
Tt c cc b vi x l c th thc hin ng thi mt cu lnh.

@2009, Ng Vn Thanh - Vin Vt L

1.6 My tnh nhiu b vi x l (multiprocessor).

My tnh nhiu b vi x l c t nht l 2 b vi x l c kt ni vi nhau

c th thc hin ng thi cc cng vic khc nhau.
Phn loi theo kiu giao tip:
Nhiu b vi x l (2-256)
Nhiu my tnh (8-256)
Phn loi theo kiu kt ni vt l:
Kt ni trn Bus (2-32)
Kt ni qua mng (8-256)
Phn loi kiu Flynn: phn loi theo kh nng x l
cc cu lnh v d liu
My tnh mt dng lnh, mt dng d liu (SISD)
My tnh mt dng lnh, nhiu dng d liu (SIMD)
My tnh nhiu dng lnh, mt dng d liu (MISD)
My tnh nhiu dng lnh, nhiu dng d liu (MIMD)
@2009, Ng Vn Thanh - Vin Vt L

Kt ni trn mt bus

@2009, Ng Vn Thanh - Vin Vt L

Kt ni y trn nhiu bus

@2009, Ng Vn Thanh - Vin Vt L

Kt ni n trn nhiu bus

@2009, Ng Vn Thanh - Vin Vt L

Kt ni tng phn trn nhiu bus

@2009, Ng Vn Thanh - Vin Vt L

Kt ni phn lp

@2009, Ng Vn Thanh - Vin Vt L

1.6.1. Nhiu b vi x l truy cp b nh ng b (Uniform Memory Access

Multiprocessors).

My tnh c kin trc kiu MIMD vi mt b nh chia s (shared memory).

Bao gm cc b vi x l c lp vi nhau.
Tt c cc b vi x l u c th truy cp n b nh chung, thi gian truy
cp n b nh ca cc b vi x l l nh nhau.
Mi mt b vi x l c th c cc thanh ghi, b m, b nh ring ca mnh.
Cc b vi x l kt ni vi nhau thng qua kiu kt ni mng (interconnection
network),
kt ni kiu mt Bus
kt ni kiu nhiu Bus
Kt ni kiu thanh ngang
Kt ni kiu b nh a cng
Nhc im: gp kh khn trong vic nng
cp b nh khi gii quyt cc bi ton ln.
@2009, Ng Vn Thanh - Vin Vt L

1.6.2 Nhiu b vi x l truy cp b nh khng ng b (Non Uniform

Memory Access Multiprocessors).

Tng t nh kin trc nhiu b vi x l truy cp b nh ng b.

Mi mt b vi x l c gn vi mt b nh ring.
u im: tng tc truy cp vao b nh.
Nhc im: thi gian giao tip gia cc b vi x l ph thuc vo tc
truyn qua mng

@2009, Ng Vn Thanh - Vin Vt L

1.7 H thng nhiu my tnh (Multicomputers)

H thng cluster: cc my tnh c lp c kt ni vi nhau thng qua mt

h thng kt ni tc cao.
Mi my tnh trong h cluster c gi l mt node.
Cc my tnh c lp c th l cc my PC thng thng c mt hoc nhiu
b vi x l.
H thng kt ni cc my tnh vi nhau dng mng LAN bng cp thng
hoc cp quang.
H thng cluster c cc u im:
Tc cao: Vi s h tr ca cc phn mm tnh ton song song, cluster
c kh nng x l mnh hn bt c mt my tnh n l no.
D nng cp: c th nng cp cho tng node hoc thm cc node.
tin cy cao: H thng vn hot ng n nh khi c mt nt (node)
trong h thng b h hng.
Gi thnh thp: v cluster c kt ni bi cc my tnh PC thng thng.
@2009, Ng Vn Thanh - Vin Vt L

1.7 H thng nhiu my tnh (Multicomputers)

H thng cluster: cc my tnh c lp c kt ni vi nhau thng qua mt

h thng kt ni tc cao.
Mi my tnh trong h cluster c gi l mt node.
Cc my tnh c lp c th l cc my PC thng thng c mt hoc nhiu
b vi x l.
H thng kt ni cc my tnh vi nhau dng mng LAN bng cp thng
hoc cp quang.
H thng cluster c cc u im:
Tc cao: Vi s h tr ca cc phn mm tnh ton song song, cluster
c kh nng x l mnh hn bt c mt my tnh n l no.
D nng cp: c th nng cp cho tng node hoc thm cc node.
tin cy cao: H thng vn hot ng n nh khi c mt nt (node)
trong h thng b h hng.
Gi thnh thp: v cluster c kt ni bi cc my tnh PC thng thng.
@2009, Ng Vn Thanh - Vin Vt L

1.8 Gii thiu v my tnh CM-200 (Connection Machine 200

by Thinking Machine Corp).

Ra i vo nm 1987, kt thc
vo nm 1991.
Thuc kiu kin trc SIMD.
S processor: 2048 65536
Tc l thuyt: 31 Gflop/s

@2009, Ng Vn Thanh - Vin Vt L

@2009, Ng Vn Thanh - Vin Vt L

@2009, Ng Vn Thanh - Vin Vt L