Parallel Processing - Xu Ly Song Song PDF

7/17/2010
1
X L SONG SONG
PARALLEL PROCESSING
THNG TIN V HC PHN
1. Thng tin chung:
1.1. Tn hc phn: X l song song (Parallel Processing)
1.2. M hc phn: KH.KM.515
1.3. S tn ch: 2 TC
1.4. Loi hc phn: Bt buc
1.5. Cc hc phn tin quyt: C s d liu phn tn
THNG TIN V HC PHN
2. Mc tiu ca hc phn:
2.1. Kin thc: cung cp cho ngi hc cc kin thc
v my tnh song song, cch xy dng cc thut ton
song song.
2.2. K nng: s dng cng c lp trnh song song nh
MPI, JAVA, VPM ... ngi hc phi ci t c mt s
thut ton song song c bn.
2.3. Thi hc tp: ngi hc phi tham d y
cc gi l thuyt v tho lun.
THNG TIN V HC PHN
3. Chnh sch i vi hc phn
Tham gia hc tp trn lp: i hc y , tch cc tho
lun ni dung bi ging, tham gia cha bi tp v chun
b bi v tt - 20%.
Kh nng t hc, t nghin cu: hon thnh bi tiu
lun (assignment) theo tng c nhn. Bi kim tra nh
gi gia k - 20%.
Kt qu thi cui k - 60%.
THNG TIN V HC PHN
3. Phn b s tit:
L thuyt: 20
Tiu lun, c thm: 8
Tho lun: 2
NI DUNG CHNG TRNH
PHN 1: TNH TON SONG SONG
Chng 1 KIN TRC V CC LOI MY TINH SONG SONG
Chng 2 CC THNH PHN CA MY TINH SONG SONG
Chng 3 GII THIU V LP TRNH SONG SONG
Chng 4 CC M HNH LP TRNH SONG SONG
Chng 5 THUT TON SONG SONG
PHN 2: X L SONG SONG CC C S D LIU
(c thm)
Chng 6 TNG QUAN V C S D LIU SONG SONG
Chng 7 TI U HA TRUY VN SONG SONG
Chng 8 LP LCH TI U CHO CU TRUY VN SONG SONG
7/17/2010
2
TI LIU THAM KHO
[0] on vn Ban, Nguyn Mu Hn, X l song song v phn tn,
NXB KH&KT, 2006.
[1] Ananth Grama, Anshui Guptal George Karipis, Vipin Kumar,
Introduction to Parallel Computing, Pearson, 2003
[2] Barry Wilkingson, Michael Allen, Parallel Programming,
Technigues and Applications Using Networked Workstations and
Parallel Computers, Prentice Hall New Jersey, 1999
[3] M. Sasikumar, Dinesh Shikhare, P. Ravi Prakash, Introduction to
Parallel Processing, Prentice - Hall, 2000
[4] Seyed H. Roosta, Parallel Processing and Parallel Algorithms,
Theory and Computation, Springer 1999.
[5] Michael J. Quinn, Parallel Computing Theory and Practice,
MaGraw-Hill,1994
[6] Shaharuddin Salleh, Albert Y. Zomaya, Scheduling in Parallel
Computing Systems, Kluwer Academic Publisher, 1999.
TI LIU THAM KHO
[7] Clement T.Yu, Weiyi Meng, Principles of Database Query
Processing for Advanced Applications, Morgan Kaufman Inc.,
1998. 185-225.
[8] Hasan Waqar, Optimization of SQL Query for Parallel
Machines, Springer, 1995.
[9] Hong W., Parallel Query Processing Using Shared Memory
Multiprocessors and Disk Arrays, Univesity of California, 1992.
[10] Hua, K.A., Parallel Database Technology, University of
Central Florida Orlande FL 32846-2362, 1997.
[11] Zomaya A. Y. and Shaharuddin Salleh, Scheduling in
parallel computing systems, Kluwer Academic Publishers, 1999.
A CH LIN H
TS. NGUYN MU HN
KHOA CNG NGH THNG TIN
TRNG I HC KHOA HC - I HC HU
77, NGUYN HU HU
IN THOI:
CQ: 054 382 6767
D: 01255213579
EMAIL: nmhan2005@yahoo.com
back
10
PHN 1:
TNH TON SONG SONG
Nguyn Mu Hn
Khoa CNTT-HKH HU
nmhan2005@yahoo.com | nmhan@hueuni.edu.vn
11
CHNG 1. KIN TRC CC LOI MY TNH SONG SONG
NI DUNG
1.1 Gii thiu chung
1.2 Kin trc my tnh kiu Von Neumann
1.3 Phn loi my tnh song song
1.4 Kin trc my tnh song song
12
1.1 Gii thiu chung
X l song song (XLSS) l g?
Trong x l tun t:
Bi ton c tch thnh mt chui cc cu lnh ri rc
Cc cu lnh c thc hin mt cch tun t
Ti mi thi im ch thc hin c mt cu lnh
7/17/2010
3
13
1.1 Gii thiu chung (tt)
1 CPU
n gin
Chm qu !!!
14
Trong x l song song
Bi ton c tch thnh nhiu phn v c th thc hin
ng thi.
Mi phn c tch thnh cc lnh ri rc
Mi lnh c thc hin t nhng CPU khc nhau
15
Nhiu CPU
Phc tp hn
Nhanh hn !!!
16
XLSS l mt qu trnh x l gm nhiu tin trnh
c kch hot ng thi v cng tham gia gii
quyt mt vn trn h thng c nhiu b x l.
Vy x l song song l g?
17
Ti sao phi x l song song?
Yu cu ca ngi s dng:
Cn thc hin mt khi lng ln cng vic
Thi gian x l phi nhanh
Yu cu thc t:
Trong thc t khng tn ti my tnh c b nh v hn
v kh nng tnh ton v hn.
Trong thc t c nhiu bi ton m my tnh x l tun t
(XLTT) kiu von Neumann khng p ng c.
S dng h thng nhiu BXL thc hin nhng tnh
ton nhanh hn nhng h n BXL.
Gii quyt c nhng bi ton ln hn, phc tp hn
18
S khc nhau c bn gia XLSS v XLTT :
X l tun t X l song song
Mi thi im ch thc hin
c mt php ton
Mi thi im c th thc
hin c nhiu php ton
Thi gian thc hin php
ton chm
Thi gian thc hin php
ton nhanh
7/17/2010
4
19
i tng no s dng my tnh song song?
20
Tnh kh thi ca vic XLSS?
Tc x l ca cc BXL theo kiu Von Neumann b gii
hn, khng th ci tin thm c.
Gi thnh ca phn cng (CPU) gim, to iu kin xy
dng nhng h thng c nhiu BXL vi gi c hp l.
S pht trin cng ngh mch tch hp cao VLSI (very large
scale integration) cho php to ra nhng h phc hp c
hng triu transistor trn mt chip.
21
Nhng thnh phn lin quan n vn XLSS:
Kin trc my tnh song song
Phn mm h thng (h iu hnh),
Thut ton song song
Ngn ng lp trnh song song, v.v.
nh ngha my tnh song song (MTSS):
MTSS l mt tp cc BXL (thng l cng mt loi) kt
ni vi nhau theo mt kiu no c th hp tc vi nhau
cng hot ng v trao i d liu vi nhau.
22
Tiu ch nh gi mt thut ton song song
i vi thut ton tun t
thi gian thc hin thut ton.
khng gian b nh.
kh nng lp trnh.
i vi thut ton song song
cc tiu chun nh thut ton tun t.
nhng tham s v s BXL: s BXL, tc x l.
kh nng ca cc b nh cc b.
s truyn thng.
thao tc I/O.
23
John von Neumann (1903 1957) : nh ton hc ngi Hungary
S dng khi nim lu tr chng trnh (stored-program concept)
Von Neumann computer c cc c im sau:
B nh c dng lu tr chng trnh v d liu
Chng trnh c m ho (code) my tnh c th hiu c
D liu l nhng thng tin n gin c s dng bi chng trnh
CPU np (fetch) nhng lnh v d liu t b nh, gii m (decode) v thc
hin tun t chng.
24
My tnh kiu V.Neumann c xy dng t cc khi c s:
B nh: lu tr d liu
Cc n v logic v s hc ALU: thc hin cc php ton
Cc phn t x l: iu khin CU v truyn d liu I/O
ng truyn d liu: BUS
B nh
B x l
Ghi d liu c d liu Cu lnh
7/17/2010
5
25
Tiu ch phn loi my tnh song song?
a) Da trn lnh, dng d liu v cu trc b nh
(Flynn)
b) Da trn kin trc: (xem 1.4)
Pipelined Computers
Dataflow Architectures
Data Parallel Systems
Multiprocessors
Multicomputers
26
Michael Flynn (1966)
SISD: Single Instruction Stream, Single Data Stream
n lung lnh, n lung d liu
SIMD: Single Instruction Stream, Multiple Data Stream
n lung lnh, a lung d liu
MISD: Multiple Instruction Stream, Single Data Stream
a lung lnh, n lung d liu
MIMD: Multiple Instruction Stream, Multiple Data Stream
a lung lnh, a lung d liu
27
M hnh SISD - n lung lnh, n lung d liu
c im
Ch c mt CPU
mi thi im ch thc hin mt lnh v ch c/ghi
mt mc d liu
C mt thanh ghi, gi l b m chng trnh (program
counter), c s dng np a ch ca lnh tip theo
khi x l tun t
Cc cu lnh c thc hin theo mt th t xc nh
y chnh l m hnh my tnh truyn thng kiu Von Neumann
28
n v iu
khin
B nh
BXL s hc
Lung lnh
Lung
d liu
Lung
kt qu
Tn hiu iu
khin
M hnh SISD - n lung lnh, n lung d liu (tt)
V d minh ha
29
M hnh SIMD - n lung lnh, a lung d liu
C mt n v iu khin (CU) iu khin nhiu n v
x l (PE)
CU pht sinh tn hiu iu khin n cc n v x l
n lung lnh: cc n v x l thc hin cng mt lnh
trn cc mc d liu khc nhau
a lung d liu: mi n v x l c lung d liu ring
y l kiu tnh ton lp li cc n v s hc trong CPU,
cho php nhng n v khc nhau thc hin trn nhng
ton hng khc nhau, nhng thc hin cng mt lnh.
My tnh SIMD c th h tr x l kiu vector, trong c
th gn cc phn t ca vector cho cc phn t x l
tnh ton ng thi.
30
M hnh ca kin trc SIMD vi b nh phn tn
M hnh SIMD - n lung lnh, a lung d liu (tt)
CU
PU1 LM1
PUn LMn
DS
DS
DS
DS
IS
IS
Program loaded
from host
Data sets
loaded from host
IS: Instruction Stream PU : Processing Unit
LM : Local Memory DS : Data Stream
7/17/2010
6
31
Cc my tnh trn th trng c sn xut theo m hnh
SIMD: ILLIAC IV, DAP v Connection Machine CM-2
M hnh SIMD - n lung lnh, a lung d liu (tt)
32
M hnh MISD - a lung lnh, n lung d liu
c im:
a lung lnh: c th thc hin nhiu lnh trn cng mt mc
d liu
n lung d liu: cc PU x l trn cng mt lung d liu
Kin trc kiu ny c th chia thnh hai nhm:
Cc my tnh yu cu mi n v x l (PU) nhn nhng
lnh khc nhau thc hin trn cng mt mc d liu.
Cc my tnh c cc lung d liu c chuyn tun t
theo dy cc CPU lin tip-gi l kin trc hnh ng-x l
theo vector thng qua mt dy cc bc, trong mi
bc thc hin mt chc nng v sau chuyn kt qu
cho PU thc hin bc tip theo.
33
M hnh MISD a lung lnh, n lung d liu (tt)
Memory:
(Program,
Data)
PU1 PU2 PUn
CU1 CU2 CUn
DS DS DS
IS
IS
IS
IS
IS
DS
I/O
MISD architecture (the systolic array)
IS: Instruction Stream PU : Processing Unit CU : Control Unit
34
M hnh MISD a lung lnh, n lung d liu (tt)
V d minh ha
35
M hnh MIMD - a lung lnh, a lung d liu
Mi BXL c th thc hin nhng lung lnh (chng trnh)
khc nhau trn cc lung d liu ring.
Hu ht cc h thng MIMD u c b nh ring v cng c
th truy cp vo c b nh chung (global) khi cn, do vy
gim thiu c s trao i gia cc BXL trong h thng.
Nhn xt:
y l kin trc phc tp nht, nhng n l m hnh h tr
x l song song cao nht
Cc my tnh c sn xut theo kin trc ny:
BBN Butterfly, Alliant FX, iSPC ca Intel
36
M hnh MIMD a lung lnh, a lung d liu (tt)
CU1 PU1
Shared
Memory
IS
IS DS
I/O
CUn PUn
IS DS
I/O
IS
MIMD architecture with shared memory
7/17/2010
7
37
M hnh MIMD a lung lnh, a lung d liu (tt)
Nhn xt:
MIMD l kin trc phc tp nht, nhng n l m hnh h tr x l song
song cao nht
Cc my tnh c sn xut theo kin trc ny:
BBN Butterfly, Alliant FX, iSPC ca Intel 38
Mt vi nhn xt:
Theo Flynn: c hai h kin trc quan trng cho cc my tnh
song song: SIMD v MIMD. Nhng kin trc khc c th
xp theo hai mu .
Mc tiu ca x l song song l khai thc n mc ti a
cc kh nng s dng ca cc thit b phn cng nhm gii
quyt nhanh nhng bi ton t ra trong thc t.
Kin trc phn cng l trong sut i vi ngi lp trnh
Trong kin trc tun t c th tn dng tc cc nhanh
ca BXL thc hin x l song song theo nguyn l chia
s thi gian v chia s ti nguyn.
Nhng chng trnh song song trn my n BXL c th
thc hin c nu c HH cho php nhiu tin trnh cng
thc hin, ngha l c th xem h thng nh l a b x l.
39
Song song ha trong my tnh tun t:
a. a n v chc nng:
Cc my tnh truyn thng ch c mt n v s hc v
logic (ALU) trong BXL. mi thi im n ch c th thc
hin mt chc nng.
My tnh song song c nhiu n v x l (PE). Nhng n
v ny c th cng nhau thc hin song song.
V d: my CDC 6600 (1964) c 10 PE c t chc trong
mt BXL. Nhng n v chc nng ny c lp vi nhau v
c th thc hin ng thi.
Xy dng b lp lch ti u phn chia cc cu lnh thc
hin sao cho tn dng c ti a cc n v x l cng
nh cc ti nguyn ca my tnh.
40
Song song ha trong my tnh tun t (tt):
b. X l theo nguyn l hnh ng trong CPU
Cu lnh c chia thnh cc giai on (stage-phase)
Ti mt thi im c th c nhiu lnh c ti vo v c
thc hin trong nhng bc khc nhau
Cc giai on thc hin khc nhau ca mi cu lnh c th
thc hin gi u nhau.
u ra ca giai on ny c th l u vo ca giai on
tip theo
Thc hin theo nguyn l hnh ng s hiu qu hn v
khng cn vng m d liu.
41
b. X l theo nguyn l hnh ng trong CPU
V d:
Pha 1: np cu lnh v t b nh (Instruction Fetch)
Pha 2: gii m (Instruction decode)
Pha 3: xc nh cc ton hng (Operand Fetch)
Pha 4: thc hin cu lnh (Instruction Execute)
Pha 5: lu tr kt qu (Write-Back)
... qu trnh ny c th phn cho mi PE thc hin mt
cng vic. Theo cch , ti mt thi im BXL c th
thc hin c nhiu cu lnh gi u nhau. Trc khi
mt cu lnh kt thc thc hin th cu lnh tip theo
c th thc hin pha gii m, cu lnh khc li c th
c np v, v.v. 42
IF: Instruction Fetch
ID: Instruction decode
OF: Operand Fetch
IE: Instruction Execute
WB: Write-Back
Instruction i IF ID OF IE
IF ID OF IE
IF ID OF IE
IF ID OF IE
IF ID OF IE
Instruction i+1
Instruction i+2
Instruction i+3
Instruction i+4
Instruction # 1 2 3 4 5 6 7 8
WB
WB
WB
WB
WB
Cycles
7/17/2010
8
43
V d: Thc hin tun t v hnh ng ca 2 tin trnh gm 4 giai on
Pha 1 Pha 2 Pha 3 Pha 4
Pha 1 Pha 2 Pha 3 Pha 4 Pha 1 Pha 2 Pha 3 Pha 4
Gi s mt tin trnh c chia thnh 4 giai on:
Thc hin tun t 2 tin trnh phi qua 8 giai on:
Thc hin theo hnh ng ch cn tri qua 5 giai on:
Tng thi gian tnh ton tun t l: 2 * (S1 + S2 + S3 + S4)
Tng thi gian tnh ton hnh ng l: S1 + S2 + S3 + S4 + S4
44
c. S gi u CPU v cc thao tc vo/ra (I/O)
Cc php vo/ra c th thc hin ng thi i vi
nhiu nhim v tnh ton khc nhau bng cch s
dng nhng b iu khin vo/ra, cc knh hay
nhng BXL vo/ra khc nhau.
Nhiu my tnh hin nay c nhiu b iu khin
thit b vo/ra, cho php a x l vo/ra lm tng
c tc trao i d liu gia cc thit b ngoi
vi CPU.
45
d. Cc h thng b nh
phn cp
Do tc thc hin cc
php ton trong BXL nhanh
hn rt nhiu vic c d
liu vo b nh trong
Cc thanh ghi c s dng
trc tip cho ALU nn b
nh cache c xem nh
vng m gia BXL v b
nh chnh
Khi d liu c chuyn t
b nh cache vo b nh
chnh th ng thi c th
chuyn d liu t cache vo
cho CPU x l
CPU
(Registers)
Cache
Main Memory
Fixed Disks
Magnetic
Tapes
Tng kh
nng lu
tr
Tng v tc
truy cp
46
e. a chng trnh v chia s thi gian
Thc hin song song da vo h iu hnh a nhim,
phn mm a lung, a tin trnh.
H iu hnh a nhim thng gii quyt cc trng hp:
Trong cng mt khong thi gian, c nhiu tin trnh
cng truy cp vo d liu t thit b vo/ra chung
Mt tin trnh tnh ton vi cng cao c th tm
thi chim dng CPU lm vic, trong khi mt tin
trnh khc trc khng i hi phi kt thc cng vic
sm phi ngng li.
B lp lch chia s thi gian lm nhim v phn chia
CPU cho mi tin trnh mt khong thi gian c nh
To cc BXL o: mi tin trnh c cung cp mt mi
trng c xem nh mt BXL thc hin ring cho
tin trnh .
47
M hnh tru tng ca my tnh song song
Mc ch: mun th hin c nhng kh nng tnh ton ca
MTSS m khng quan tm n nhng rng buc c th ca
nhng my tnh c trong thc t.
Ch : khi xy dng cc thut ton song song, chng ta qui
c l pht trin thut ton cho m hnh tru tng ny,
sau nh x sang nhng my tnh c th vi mt s cc
rng buc no .
48
a. My tnh truy cp ngu nhin song song PRAM
Cha mt n v iu khin CU
Mt b nh chung
Mt tp khng gii hn cc BXL
Mi BXL li c b nh ring v c mt ch s duy nht
c s dng xc nh a ch trong qu trnh trao
i cc tn hiu v qun l cc ngt.
Tt c cc BXL u chia s b nh chung vi yu cu
khng b gii hn.
Cc cu lnh c th bt u thc hin bt k thi im
no, bt k v tr no ca b nh (ring hoc chung)
7/17/2010
9
49
a. My tnh truy cp ngu nhin song song PRAM
y cng l m hnh tng qut cho my tnh song song kiu MIMD
Private memory
P
1
Private memory
P
2
Private memory
P
n
Interconnection network
Global memory
CU
50
Mt s iu cn lu khi pht trin nhng thut ton
cho cc MTSS tng qut
Khng b gii hn v s lng BXL
Mi v tr ca b nh u truy cp c bi bt k BXL
no
Khng gii hn v dung lng b nh chung chia s
trong h thng
Cc BXL c th c bt k mt v tr no ca b nh,
ngha l khng cn phi ch nhng BXL khc kt
thc cng vic truy cp vo b nh.
51
Mt s iu cn lu khi chuyn nhng thut ton xy
dng cho MTSS tng qut sang my tnh c th
Phi p dng mt s cc rng buc m bo chng
trnh thc hin c trn nhng my tnh .
V hnh thc, phi thc hin mt trong nhng iu kin
sau:
EREW: loi tr vn xung t c/ghi
(Exclusive Read + Exclusive Write)
CREW: cho php c ng thi, nhng khng cho
php xung t khi ghi
(Concurrent Read + Exclusive Write)
CRCW: Cho php c, ghi ng thi
(Concurrent Read + Concurrent Write)
52
b. Kin trc SIMD
Cu trc:
Cc phn t x l (PE) u c iu hnh bi mt
n v iu khin (CU).
Cc phn t x l nhn c cng mt lnh t CU
nhng hot ng trn nhng tp d liu khc nhau.
c tnh:
Phn tn vic x l trn nhiu phn cng
Thao tc ng thi trn nhiu phn t d liu
Thc hin cng mt tnh ton trn tt c cc phn t d
liu.
53
b. Kin trc SIMD (tt)
CU
PE
2
IS
PE
n
PE
1
.
.
.
Global memory
Result
IS
DS
n
DS
2
DS
1
M hnh kin trc
kiu SIMD
IS: Instruction Stream PE : Processing Element
54
V d
X
1
X
2
X
4
X
3 X=
No
Yes
X
1
X
3
X
4
X
2
Mt s PE kim tra X= ,
mt s khc ri
Mt s PE kim tra X= ,
mt s khc ri
Tt c PE
(a) thc hin theo SISD,
(b) thc hin theo SIMD
Trong , X
1
, X
2
, X
3
, v X
4
l cc
khi cc cu lnh
7/17/2010
10
55
b. Kin trc MISD
BXL hnh ng chnh l BXL kiu MISD
Nguyn l hnh ng (pipeline): da trn nguyn tc:
Phn on hoc chia nh mt tin trnh thnh mt s tin
con thc hin trong cc pha lin tip.
Mi mt giai on ca mt tin trnh c thc hin tun t.
Sau khi thc hin xong mt pha th bt u thc hin giai
on ca tin trnh tip theo.
Mi pha thc hin xong s truyn kt qu cho pha tip theo.
Tm li, theo nguyn l hnh ng:
Khi mt giai on cng vic ang thc hin th mt giai
on khc c th np d liu vo, v d liu vo ca giai
on ny c th l kt qu ca giai on trc n.
56
V d: Thc hin tun t v hnh ng ca 2 tin trnh gm 4 giai on
Pha 1 Pha 2 Pha 3 Pha 4 Pha 1 Pha 2 Pha 3 Pha 4
Gi s mt tin trnh c chia thnh 4 giai on:
Thc hin tun t 2 tin trnh phi qua 8 giai on:
Thc hin theo hnh ng ch cn tri qua 5 giai on:
Tng thi gian tnh ton tun t l: 2 * (S1 + S2 + S3+ S4)
Tng thi gian tnh ton hnh ng l: S1 + S2 + S3 + S4 + S4
57
Nguyn l hnh ng c th p dng theo hai mc:
- Hnh ng theo n v s hc:
Cc n v s hc v logic ALU c t chc thnh mng,
cc php ton bn trong c thc hin theo nguyn l
hnh ng.
CU
ALU
ALU
. . .
ALU
B nh
X l hnh ng theo ALU
58
- Hnh ng theo n v cu lnh:
Cc n v iu khin CU c phn on v t chc theo
hnh ng.
CU
. . .
CU
CU
ALU
B nh
X l hnh ng theo CU
59
Xy dng hnh ng vng trn gia cc BXL, b nh v mng
lin kt
Php ton thc hin bi CU theo kin trc ny c th chia thnh 5 giai on:
G1: c d liu: c d liu t b nh chia s (Shared Memory).
G2: Chuyn ti d liu: chuyn d liu t b nh ti cc phn t x l PE
thng qua mng c (Read Network).
G3. Thc hin cu lnh: s dng PE thc hin cc cu lnh.
G4. Chuyn ti d liu: chuyn cc kt qu t cc PE ti b nh
thng qua mng ghi (Write Network).
G5. Lu tr d liu : ghi li cc kt qu vo b nh chia s.
Shared
Memory
Shared
Memory
Write
Network
Read
Network
V d v mt hnh ng vng trn
60
Mt v d n gin
Tnh tng n phn t ca mt mng a:
For (i=0; i<n; i
++
)
s=s+a[i]
CPU
s=s+a[i]
s
a[0] a[1] a[2] a[3] a[4]
Thc hin bnh thng
read
read
write
7/17/2010
11
61
Tnh tng n phn t ca mt mng a:
s
in
= s
out
+ a[i]
s
in
s
out
a[0] a[1] a[2] a[3] a[4]
Thc hin theo ng ng
s
in
s
out
s
in
s
out
s
in
s
out
s
in
s
out
62
V d: Sp xp n phn t bng k thut ng ng
Recv(&number, P
i-1
);
If (number > x) {
send(&x,P
i-1
);
x= number;
} else send(number, P
i+1
);
tng thut ton (insertion sort):
P
0
nhn mt dy cc s
Lu tr s ln nht
Chuyn cc s nh hn i
Nu s nhn c ln hn s lu tr hin thi th thay th
n v chuyn s nh hn i
63
4,3,1,2,5
5
5
5 2
5 3 1
5 4 2
5 4 3 1
5 4 3 2
5 4 3 2 1
P
0
P
1
P
2
P
3
P
4
4,3,1,2
4,3,1
2
4,3
1
1
4
5 2
3
4 2
3 1
2
1
64
c. B x l mng tm thu SAP (Systolic Array Processor)
Do Kung v Leiserson xut 1978
L mt mng cc phn t x l d liu (DPU) c kt ni
cc b vi nhau.
Mi DPU c xem nh mt t bo, mt trong mng, bao
gm:
Mt s thanh ghi (registers)
Mt b cng (adder)
Cc mch iu khin
n v logic-s hc ALU.
65
c. B x l mng tm thu SAP (cont.)
Systolic
Array Controller
Host
Processor
Tn hiu
D liu vo
Kt qu
Kin trc b x l mng tm thu
Trong kin trc SAP,
B iu khin (Controller) lm nhim v giao din cho BXL
chnh (Host Processor) v gi cc tn hiu iu khin qu
trnh vo/ra d liu cho SA.
Hot ng ca h thng theo tng nhp v lp li mt cch
u n tn dng c kh nng song song ca tt c
cc phn t x l
66
c. B x l mng tm thu SAP (tt)
D
P
U
:

D
a
t
a

p
r
o
c
e
s
s
i
n
g

U
n
i
t
s
7/17/2010
12
67
c. B x l mng tm thu SAP (cont.)
SA c th t chc theo nhiu cu hnh tp khc nhau
(a) mng tuyn tnh
(b) mng hnh tam gic
(c) mng hai chiu hnh vung
68
c. B x l mng tm thu SAP (tt)
Xt bi ton nhn 2 ma trn c 2x2: AxB=C
S dng b nh SAP hai chiu hnh vung tnh
22 22 12 21 22 21 22 11 21 21
22 12 12 11 12 21 12 11 11 11
22 21
12 11
22 21
12 11
22 21
12 11
* * * *
* * * *
*
*
b a b a c b a b a c
b a b a c b a b a c
b a c
c c
c c
b b
b b
a a
a a
k
kj ik ij
+ = + =
+ = + =
|
|
.
|
\
|
=
|
|
.
|
\
|
|
|
.
|
\
|
=
69
4 5 6
7 8 9
1 2 3
a
11
a
12
a
21
a
22
b
22
b
21
b
12
b
11
c
11
c
12
c
21
c
22
Nhp theo ct
Nhp theo hng
Nhp 1:
Nhp a
11
, b
11
vo s 1 v tnh a
11
* b
11
Nhp b
21
vo s 4 v a
12
vo s 2
Nhp 2:
Truyn b
11
t s 1 sang s 2
Truyn a
11
t 1 sang 4 v tnh a
11
* b
21
Truyn b
11
t 1 sang 2 v tnh a
12
* b
11
Truyn b
12
t 4 sang 5 v a
12 t
2 sang
5 v tnh a
12
* b
21
. Tnh c
11
=a
12
*b
11
+a
12
*b
21
Nhn tip b
12
vo 4 v a
21
vo s 2
Nhp 3:
Truyn c
11
t s 5 sang s 9
Truyn a
21
*b
11
t 2 sang 6
Truyn b
12
t 5 sang 6
Nhp a
22
vo 3 v nhp b
22
vo 7
Nhp 4:
Truyn a
22
t s 3 sang s 6
Tnh a
22
* b
21
Truyn a
21
*b
11
t 2 sang 6
Cng dn kt qu c cuyn t 2:
S cho c
21
=a
21
*b
11
+ a
22
* b
21
Chuyn c
11
t 9 ra v gn cho c
11
70
d. Kin trc MIMD
Loi a BXL hoc cn gi l h thng a my tnh
Trong mi BXL c n v iu khin (CU) ring v thc
hin chng trnh ring ca mnh.
Nhng c trng ca MIMD:
X l phn tn trn mt s BXL c lp
Chia s vi nhau mt s ti nguyn, trong c h thng
b nh.
Mi BXL thao tc c lp v c th thc hin ng thi vi
nhau.
Mi BXL c th thc hin mt chng trnh ring.
71
CU
1
PE
2
IS
1
PE
n
PE
1
.
.
.
CU
2
CU
n
.
.
.
DS
IS
2
IS
n
Kin trc MIMD
72
Cu hi chng 1
1. Nu c trng c bn ca kin trc my tnh tun t ca VN
2. Cc kin trc my tnh c th c phn loi nh th no?
da vo nhng yu t no phn loi?
3. Mt h thng nh th no c gi l my tnh song song?
4. My tnh kiu MIMD khc vi mng cc my tnh nh th
no?
5. Nu nguyn l x l theo hnh ng. Nhng bi ton c nhng
tnh cht g th thch hp vi kin trc x l hnh ng?
6. Cn bao nhiu nhp thc hin nhn hai ma trn 100 100
trn SAP c 100100 phn t x l? Nu s dng h
10001000 PE th h no tt hn (nhanh hn)?
7. Mt cng vic c chia thnh m cng vic con, mi cng
vic con i hi mt n v thi gian thc hin. Hi cn bao
nhiu n v thi gian h hnh ng gm m-b x l tuyn
tnh (gm m-pha thc hin) thc hin c nhim v cho
trc?
7/17/2010
13
73
CHNG 2. CC THNH PHN CA MT SONG SONG
NI DUNG
2.1 B nh ca MTSS
2.2 Mng kt ni cc thnh phn ca MTSS
2.3 Chng trnh dch v h iu hnh
2.4 Kt lun
74
2.1 B nh ca MTSS
Mt s vn cn quan tm trong kin trc MTSS
S dng nhiu thanh ghi:
s lm gim hiu ng ph ca cc ngt
S dng khng gian nh ln:
lm gim hiu ng ph ca s i ch khi thc hin mt
lnh.
tng hiu qu trao i d liu ca h thng.
Xy dng b lp lch (Scheduling): nhm cp pht hu hiu
tng nhim v n l cho cc BXL cho mt cch ng.
ng b cc BXL (Synchronization): iu khin nhiu tin trnh
hot ng ng thi, cng truy cp n mt s hu hn cc ti
nguyn dng chung, trnh c s tc nghn (deadlock)
75
2.1 B nh ca MTSS
Mt s vn cn quan tm trong kin trc MTSS (cont.)
Thit k cu hnh mng: tp trung vo vic kt ni gia BXL
vi BXL, gia BXL vi b nh trong h thng. Cu hnh tp
ca mng kt ni l vn rt quan trng trong thit k h
thng song song.
Phn on( Partitioning): xc nh mc song song trong
cc thut ton xc nh c cc lung x l ng thi.
S phn on c th th hin nhiu mc khc nhau: mc
lnh, mc th tc, hoc mc chng trnh, v.v.
m bo tin cy (Reliability): nu c mt BXL no khng
thc hin c cng vic m nhim th mt BXL khc s
c thay th m bo trong mi tnh hung cng vic
chung vn c hon thnh.
76
2.1 B nh ca MTSS
B nh ca MTSS c chia thnh nhiu mc
B nh mc 1: mc cao nht, c dung lng nh nht, truy
cp nhanh v t nht, thng gn cht vi mi BXL thnh
b nh cc b (local memory-khc vi b nh chia s).
B nh mc 2: Truy cp chm hn v r hn mc 1, v.v.
V nguyn tc, d liu c chuyn i gia cc mc ln
cn ca cc b nh v hon ton c iu khin bi b nh
mc 1.
V l thuyt, trong cu trc phn cp b nh, tc truy cp
b nh trung bnh gn bng tc truy cp mc cao nht
(mc 1), nhng chi ph ca cc n v nh trung bnh li gn
vi gi ca b nh mc thp nht (mc n).
77
BXL
B nh mc 1
B nh mc n
B nh mc 2
Mc thp nht
Mc cao nht
Nh, nhanh v t
Ln, chm v r
2.1 B nh ca MTSS
Phn cp h thng b nh
78
2.1 B nh ca MTSS
2.1.1 B nh kt hp (AM Associative Memory)
AM bao gm cc nh (cell)
s
q
k
a
m
R/W
Select
Key
Argument
Read/Write
Output
Match
Mi nh ca AM c 4 u vo v hai u ra
Cc u vo (input) ca mi nh bao gm: . Bit i s a,
. Bit c/ghi R/W xc nh thao tc tng ng cn thc hin
. Bit kho k
. Bit la chn s xc nh nh thch hp cho vic thc hin c/ghi.
Hai kt qu u ra:
. Bit i snh m ch ra d liu c lu trong b nh c i snh c
vi bit i s a.
. Bit kt qu ra q.
7/17/2010
14
79
2.1 B nh ca MTSS - Cu trc ca b nh kt hp
0
1
.
.
.
m-1
Argument Register Mask Register
Buffer Register
Tags
Match
Register
Input
Input
Output
Array of memory locations
0 1 . . . n-1
80
2.1 B nh ca MTSS
V d, tm trong AM tt c cc t c 8 bit cao nht cha gi tr
(1101 1100)
2
v tr li t u tin c tm thy.
Gi tr (1101 1100)
2
l i s tm
Thanh ghi nh du (mask register) l 8 bit cao nht. Qu
trnh tm kim thc hin nh sau:
1. T cn tm (1101 1100)
2
c np vo thanh ghi i s
Argument Register
2. on m chng ta quan tm l 8 bit cao nht, nhng bit
ny c a vo thanh ghi nh du Mask Register
nh du.
3. Tt c cc t trong b nh c so snh song song vi
thanh ghi i s.
81
2.1 B nh ca MTSS
2.1.2 M hnh b nh truy cp ngu nhin song song
B nh PRAM gm:
m vng b nh ln chia s cho p b x l.
c s dng lu tr d liu v l vng trao i d liu
gia cc BXL.
cc BXL c th truy cp vo b nh PRAM ng thi v c th
hot ng mt cch d b.
V d, b x l P
i
ghi d liu vo mt vng nh v d liu ny c
th c P
j
truy cp, ngha l P
i
v P
j
c th dng b nh chung
trao i vi nhau.
M hnh loi ny c cc phng thc truy cp sau:
82
2.1 B nh ca MTSS
Cc phng thc truy cp b nh
Concurrent Read (CR): nhiu BXL c th c ng thi cng
mt nh.
Exlusive Read (ER): p BXL c c p vng nh khc nhau
v mi BXL c c duy nht mt vng nh v mi vng nh
c c bi mt BXL.
Concurrent Write (CW): cng mt thi im cho php nhiu
BXL ghi vo cng mt vng nh.
Exlusive Write (EW): p BXL ghi c vo p vng nh khc
nhau. Mi BXL ghi c vo mt vng nh v mi vng nh
c ghi bi mt BXL.
Ngi ta phn CW thnh cc loi nh sau:
83
2.1 B nh ca MTSS
Ghi ng thi c u tin (Priority CW): cc BXL c gn mc u
tin v khi c nhiu BXL mun ghi d liu vo mt vng nh th u
tin cho BXL c mc u tin cao nht. Cc mc u tin c th gn
tnh hoc ng theo nhng qui tc c xc nh khi thc hin.
Ghi ng thi chung (Common CW): Khi cc BXL ghi cng mt gi
tr th chng c php ghi vo cng mt vng nh. Trong trng
hp ny, mt BXL s c chn ghi d liu .
Ghi ng thi t do (Arbitrary CW): mt s BXL mun ghi d liu
ng thi vo mt vng nh, nhng c mt BXL c php thay i
gi tr ca vng nh . Trong trng hp ny, chng ta phi ch ra
cch la chn BXL thc hin vic ghi.
Ghi ng thi ngu nhin (Random CW): BXL c la chn
ghi d liu l ngu nhin.
Ghi ng thi t hp (Combining CW): tt c cc d liu m cc
BXL nh ghi ng thi vo b nh c t hp li thnh mt gi tr.
Gi tr ny s c ghi vo b nh .
84
2.1 B nh ca MTSS
2.1.2 B nh chia s (Share Memory)
c trng:
Dung lng ln
Cc BXL u c th truy cp vo SM
Cc BXL c th hot ng c lp nhng lun chia s a ch
cc nh (khng c c quyn)
Nhng thay i ni dung nh c thc hin bi mt BXL
no s c nhn thy bi cc BXL khc.
Cc my tnh s dng b nh chia s c phn lm 2 loi:
UMA (UniformMemory Access)
NUMA (NonUniformMemory Access)
Share Memory
7/17/2010
15
85
2.1 B nh ca MTSS
a. M hnh UMA ca b nh chia s
c im:
Cc BXL lm vic nh c ch chuyn mch tp trung (central
switching mechanism) iu khin vic truy cp ti b nh
chia s.
Cc BXL u c th truy cp n b nh chia s ton cc
(global shared memory)
Thi gian truy cp vo b nh l nh nhau i vi mi BXL.
Mt BXL ny c th trao i thng tin vi mt BXL khc bng
cch ghi thng tin vo b nh ton cc v BXL kia s c d
liu ti cng v tr trong b nh.
86
2.1 B nh ca MTSS
a. M hnh UMA ca b nh chia s (tt)
Nu tt c cc BXL ca my tnh u c thi gian truy cp n
cc thit b l nh nhau th gi l my tnh a b x l i xng
- Symmetric MultiProcessor (SMP) machines
My tnh vi kin trc UMA cn c gi: CC-UMA - Cache
Coherent UMA
P
1
Switching mechanism
I/O
P
2

P
n
Memory banks
C1 C2 Cn
P
i
C
i
Processor i
Cache i
87
2.1 B nh ca MTSS
a. M hnh UMA ca b nh chia s (tt)
88
2.1 B nh ca MTSS
b. M hnh NUMA ca b nh chia s
c im
B nh chia s c phn tn v chia thnh cc moun nh
c lp.
B nh chia s c phn tn cho tt c cc BXL thnh b
nh cc b v tt c cc moun nh s l b nh chung cho
cc BXL.
M hnh NUMA thng c to thnh t hai hoc nhiu
SMPs ni vi li vi nhau bi mt ng truyn vt l.
Mt SMP c th truy cp trc tip n mt SMP khc
Cc BXL c php truy cp ng thi ti mt hay nhiu m
un nh v c th hot ng c lp vi nhau.
Khng phi tt c cc BXL u c thi gian truy cp n cc
b nh l nh nhau. Truy cp b nh qua link s chm hn
89
2.1 B nh ca MTSS
b. M hnh NUMA ca b nh chia s (tt)
Network
Cache
P
Mem Cache
P
Mem
Cache
P
Mem Cache
P
Mem
NUMA vi Distributed Memory
90
2.1 B nh ca MTSS
2.1.3. B nh phn tn
c im:
C mt ng kt ni gia
cc BXL
Cc BXL u c b nh ring.
a ch b nh ca mt BXL no khng c bit bi BXL
khc. Do khng c khi nim khng gian a ch tng th
(global address space) qua cc BXL khc.
V BXL ny khng th t do c/ghi vo b nh ca mt BXL
khc nn khi nim Cache coherent khng p dng y.
Khi cn thit ngi lp trnh c nhim v chuyn d liu t
nh ca BXL ny sang nh ca mt BXL khc.
7/17/2010
16
91
2.1 B nh ca MTSS
2.1.3. B nh phn tn
92
2.1 B nh ca MTSS
2.1.3. B nh a my tnh
Mi nt trong h thng a my tnh cng chnh l mt my tnh
c b nh ring, khng chia s vi nhng BXL khc.
Cc BXL trao i vi nhau thng qua vic gi v nhn cc
thng ip (messages)
Khng tn ti b nh chia s chung m mi BXL c b nh cc
b ring ca chng.
Vic trao i d liu trong mng theo m hnh point to point
thng qua s lin kt tnh gia cc BXL.
Memory BXL
BXL Memory
Memory BXL
Memory BXL
c im:
93
2.1 B nh ca MTSS
2.1.4 My tnh vi b nh lai (Hybrid Memory)
Kt hp gia kin trc chia s b nh chung v b nh phn
tn.
H thng l nhiu cm my tnh khc nhau. Mi cm l cc
my tnh vi b nh c kin trc chia s v c b nh ton cc
ring cho cm .
Cc BXL trong mt cm chia s b nh chung c th truy cp
n b nh ton cc ring ca cm .
Thnh phn b nh phn tn c hiu nh l mt cm cc
b nh ton cc ca mi cm.
Cc BXL ch c th truy cp n b nh ton cc trong thnh
phn chia s b nh phn tn ca chng, ch khng truy cp
c b nh ca cc thnh phn chia s b nh chung khc.
94
2.1 B nh ca MTSS
6. My tnh vi b nh lai
Hybrid Distributed-Shared Memory
95
2.1 B nh ca MTSS
Tm li,
Hai iu cn phi quan tm trong thit k b nh ca h thng
song song. l,
Bng thng (bandwidth): ng kt ni cc BXL, b nh
tr b nh (memory latency), tr b nh c hiu nh
l khong thi gian t khi BXL yu cu d liu t b nh n khi
BXL nhn c d liu tng ng.
Trong h thng my tnh song song bng thng phi rng
thc hin vic truyn thng ca cc BXL.
Khi cc mun nh c chia s th vic cnh tranh s dng
khng gian nh phi c cc tiu ha. Do , tr b nh
cng phi c cc tiu ha.
96
Mt vi nhn xt:
Trong hu ht cc kin trc song song nh: nhng h
a BXL chia s b nh, a b x l a b nh , v.v. th
vn quan trng nht trong thit k l xc nh s kt
ni gia cc BXL v b nh vi nhau.
Mt kin trc l tng l kin trc trong , mi BXL
u kt ni c vi cc BXL cn li. Khi n to ra
mt th y .
V d, nu h c p BXL th s c p*(p 1)/2 ng
lin kt. D nhn thy kin trc loi ny s rt phc
tp, nht l khi p ln.
7/17/2010
17
97
C hai loi cu hnh tp cho mng lin kt: lin kt
tnh v lin kt ng.
Mng lin kt tnh l mng cc thnh phn ca h
thng my tnh, trong cc BXL, b nh c lin
kt vi nhau mt cch c nh, khng thay i c.
Mng lin kt ng l mng cc thnh phn ca h
thng my tnh, trong s lin kt gia cc BXL, b
nh c th thay i c.
98
2.2.1 Lin kt tuyn tnh v vng (linear and ring)
a. Lin kt tuyn tnh
Cc BXL c lin kt vi nhau theo dy v c nh s theo
th t tng dn
P
0 P
1 P
n-1
. . .
Nhn xt:
Trong mng ln kt tuyn tnh, tr hai phn t u v cui, tt
c cc BXL bn trong u c hai lng ging l BXL trc v sau
n.
y l dng lin kt n gin, nhng d liu cng cn phi
chuyn qua nhiu BXL, do s truyn thng d liu gia cc
BXL, c bit l gia BXL u v cui s b chm li khi s BXL
kh ln.
99
2.2.1 Lin kt tuyn tnh v vng
b. Lin kt vng
c t chc tng t nh lin kt tuyn tnh nhng BXL u
v cui c ni vng vi nhau
Trong mng lin kt vng, s trao i gia cc BXL c th thc
hin theo mt chiu, gi l mng n, hoc theo c hai chiu gi
l mng kp.
Tt nhin s truyn thng trong mng lin kt vng, nht l gia
nhng BXL xa nhau th cng vn b tr.
P
2
P
6
P
5
P
3
P
4 P
0
Nhn xt:
100
2.2.2 Mng lin kt xo trn hon ho (Perfect Shuffle Exchange)
Gi s c N BXL: P
0
, P
1
, , P
N-1
, vi N l ly tha ca 2.
Trong mng lin kt xo trn hon ho, ng lin kt mt
chiu t P
i
ti P
j
c xc nh nh sau:
2i vi 0 s i s N/2 - 1
2i+1-N vi N/2 s i s N - 1
Mng lin kt xo trn hon ho mt chiu vi N = 8=2
3
,
trong s lin kt xo trn c k hiu bng mi tn.
j = {
P
0
P
1
P
2
P
3
P
4
P
5
P
6
P
7
101
Mt biu din khc ca mng lin kt xo trn hon ho
P
001
P
010
P
011
P
100
P
000
P
101
P
110
P
111
P
001
P
010
P
011
P
100
P
000
P
101
P
110
P
111
102
2.2.2 Mng lin kt xo trn hon ho hai chiu
Mng lin kt xo trn hon ho hai chiu vi N = 8,
trong s lin kt xo trn c k hiu bng mi tn hai chiu.
P
0
P
1
P
2
P
3
P
4
P
5
P
6
P
7
7/17/2010
18
103
2.2.3 Mng lin kt li hai chiu (two-dimentional mesh)
c im:
Mi BXL c lin kt vi bn lng ging: trn, di, bn tri v
bn phi.
Mng lin kt li hai chiu c s dng thit k cc my
tnh ILLIAC IV, MPP (Massively Parallel Processor), DAP (ICL
Distributed Array Processor), v.v.
C hai dng lin kt li:
(a) Li khng quay vng (b) Li quay vng
104
2.2.4 Mng lin kt siu khi hoc hnh khi n-chiu
Gi s c n b x l: P
0
, P
1
, , P
n-1
, vi n = 2
q
, q > 0. Nu mi
BXL cn lin kt vi ng q b x l ln cn th c th dng
mng lin kt theo hnh siu khi n chiu.
Trong mng lin kt hnh khi cc ch s ca cc BXL c
nh nh phn, hai BXL c gi l lng ging vi nhau nu
nhn ch s ca chng sai khc nhau ng mt bit.
Mng lin kt hnh khi vi n=8 (q=3)
2
P
000
0
5
1
4
7
6
3
P
001
P
100
P
101
P
111
P
110
P
010 P
011
105
Mng lin kt hnh khi vi n=16 (q=4)
2
0
5
1
4
7
6
3
2
0
5
1
4
7
6
3
5 = 0101
1 = 0001
4 = 0100
13 = 1101
106
2.2.5 Mng lin kt hnh sao (star)
K hiu l S
n
C n! b x l
Nhn ca mi BXL s tng ng vi mt hon v ca n k hiu.
BXL P
i
c kt ni vi BXL P
j
j nhn c t i bng cch
thay k hiu th k ca hon v n phn t 1,..., n, vi 2sksn.
P
1234
P
3214
P
2134
P
3124
P
1324
P
2314
V d, trong S
4
(n = 4) c 4! = 24 BXL.
Hai BXL P
2134
v P
3124
l c lin kt vi
nhau v 3124 nhn c t 2134 bng
cch i ch v tr u (s 2) vi v tr
th ba (s 3). Tng t suy ra P
2134
v
P
4132
l c lin kt vi nhau.
107
Mng lin kt hnh sao vi 24 b x l
P
3412
P
4312
P
1432
P
1342 P
4132
P
3142
P
4231
P
3241
P
2431
P
2341 P
3421
P
4321
P
1234
P
3214
P
2134
P
2314 P
3124
P
1324
P
2413
P
4213
P
1423
P
1243 P
4123
P
2143
108
2.2.5 Mng lin kt Butterfly
(k+1)2
k
nt c chia thnh (k+1) ranks, mi rank cha n=2
k
nt.
Cc ranks c nh s t 0 n k
K hiu Node(i,j): nt th j trn rank th i
Node(i,j) c ni vi 2 nt trn rank i-1: node(i-1,j) v node (i-1,m),
y m l s nguyn c c bng cch o bit th i trong xu nh
phn biu din j
Nu node(i,j) c ni n node(i-1,m), th node (i,m) c ni vi
node(i-1,j)
7/17/2010
19
Rank 0
Rank 1
Rank 2
Rank 3
Node(1,5): i=1, j=5
j = 5 = 101 (binary)
i=1
001 = 1
Node(1,5) c ni vi
node(0,5) v node(0,1)
0 1 2 3 4 5 6 7
Mng lin kt Butterfly vi 4 ranks v 32 BXL
2.2.5 Mng lin kt Butterfly (tt)
109
Mt thp chiu cao d gm (4
d+1
-1)/3
BXL c phn tn trong d+1 mc c
cc tnh cht sau:
C 4
d-2
BXL mc d.
C 4
d-1
BXL mc d-1, v
C 4
d
BXL mc d-2
Ch c 01 BXL mc d
i vi mc x:
c ni vi 04 nt lng ging
cng mt mc nu x<d
c ni vi 04 nt con mc x-
1 nu x>1
c ni vi 04 nt con mc
x+1 nu xd-1
2.2.6 Mng lin kt Pyramid
Mng lin kt Pyramid
C d=2 v (4
2+1
-1)/3 =21 BXL
Level 0
Level 1
Level 2
110
Chiu cao ca cy gm k mc, c nh s t 0 n k-1
Mi BXL l mt node ca cy, c N=2
k
-1 nodes
Mi BXL mc i c ni vi nt cha mc i+1 v 2 nt
con mc i-1
2.2.7 Mng lin kt nh phn (binary tree)
Mng nh phn c chiu cao k=4, s node N= 2
4
-1=15
Level 0
Level 1
Level 2
Level 3
111
2.2.8 Mng Hypertree
c im:
d l chiu cao cy c nh s t 0 n d-1 vi 4
d
nt l
C 2
d
(2
d-1
-1) nodes
Nhn t pha trc mng nh mt cy c chiu cao d
Nhn t pha bn mng nh mt cy nh phn o ngc c
chiu cao d
Front view
Side view
2.2.8 Mng Hypertree
Front view
Side view
Mng hypertree bc 4, chiu cao 2
Gm N = d
k
BXL
Ch s mi BXL biu din vi k ch s (a
k-1
a
k-2
a
j
a
1
a
0
), vi
a
j
e {0, 1,d-1} , j = 0, 1,, k-1.
s b x l c th t c bi (a
k-1
a
k-2
a
1
a
0
) l
(a
k-2
a
k-3
a
0
q) v (qa
k-1
a
k-2
a
2
a
1
), q = 0, 1,, d-1.
2.2.9 Mng lin kt De Bruijn
114
P
000
P
001
P
100
P
111
P
011
P
110
P
010
P
101
V d mt mng De Bruijn vi d = 2, v k = 3, N = 2
3
, c ba
ch s a
2
, a
1
, a
0
7/17/2010
20
C k tng.
Mi tng gm cc thanh cho c m input v n output. tng
cng m*n thanh cho.
Mng bao gm m
k
input v n
k
output; do n mng c
m
k
*
n
k
chuyn mch.
Cc kt ni chuyn mch cho php to mt ng dn
chnh xc t cc input n cc output.
Nu A l a ch ch ca mt kt ni mong mun trn c s
ca n, th cc s ca A biu din cc thanh cho t ra
thit lp kt ni theo d kin .
2.2.10 Mng lin kt Delta
115
2.2.10 Mng lin kt Delta
116
mng delta 3 tng, c 2
3
*
2
3
ng kt ni, dng 2
*
2 thanh cho.
Mi tng c 2
3
input v 2
3
output.
tng 1 tng 2 tng 3
M
0
M
1
M
2
M
3
M
4
M
5
M
6
M
7
P
0
P
1
P
2
P
3
P
4
P
5
P
6
P
7
117
2.2.8 Mt s mng kt ni khc
Tham kho:
[6] M. Sasikumar, Dinesh Shikhare, P. Ravi Prakash, Introduction to Parallel
Processing, Prentice Hall. 2002
[7] Seyed H. Roosta, Parallel Processing and Parallel Algorithms, Theory and
Computation, Springer, 2006.
[8] Michael J. Quinn, Parallel Computing Theory and Practice, MaGraw-Hill,
2004
118
2.4 Kt lun
Vn trng tm ca chng 2
1. Tm hiu cc ti nguyn v mi quan h ca chng trong h
thng tn dng c ht kh nng x l song song.
2. Cc b nh c t chc thnh b nh kt hp, b nh truy
cp ngu nhin, b nh chia s, v.v. l cc m hnh chnh cho
vic thit k b nh MTSS.
3. Vn quan trng trong thit k kin trc ca MTSS l xc
nh cch kt ni cc b x l vi nhau sao cho hiu qu
nht.
4. Cc b x l c th kt ni theo mng lin kt tnh hoc lin
kt ng.
5. Kho st mt s cu hnh tp ca mng lin kt cc b x
l nh: lin kt tuyn tnh, lin kt xo trn, lin kt li, lin
kt hnh sao v lin kt hnh khi...
119
2.3 Chng trnh dch v cc h iu hnh
Cu hi cui chng
1. Nu nhng vn cn quan tm khi thit k kin trc my
tnh song song
2. B nh kt hp l g? nu nguyn l hat ng ca b nh
kt hp.
3. Ti sao mng lin kt li ng vai tr quan trng trong kin
trc MTSS?
4. Da vo nh ngha chung ca mng lin kt hnh khi xy
dng cu trc tp cho mng lin kt hnh khi cho 16 b x
l.
5. Nu nhng c trng c bn ca chng trnh dch song
song?
6. Nu mc ch chnh ca h iu hnh cho my tnh song
song?
7. Xy dng mng lin kt theo m hnh xo trn hon ho cho
16 phn t.
120
Tiu lun
1. Kin trc kt ni mng
De Bruijn, Mng cy nh phn, Mng Delta, Mng Butterfly,
Mng Omega, Mng Pyramid ([7] Seyed H. Roosta, Parallel Processing
and Parallel Algorithms, Theory and Computation, Springer, page 72-80)
2. Chng trnh dch song song ([7] Seyed H. Roosta, Parallel
Processing and Parallel Algorithms, Theory and Computation,
Springer, page 83-90)
3. H iu hnh a x l ([7] Seyed H. Roosta, Parallel Processing and
Parallel Algorithms, Theory and Computation, Springer, page 91-100)
4. Tm hiu h iu hnh Linus-Windows (lin quan n a lung a
tin trnh)
7/17/2010
21
121
CHNG 3. GII THIU V LP TRNH SONG SONG
NI DUNG
1. Nhng khi nim c s ca lp trnh song song
2. Cc ngn ng lp trnh song song
3. Cc loi ph thuc d liu trong chng trnh
4. Phng php bin i chng trnh tun t
sang chng trnh song song
122
3.1 Gii thiu chung
S khc nhau c bn gia LT tun t v LT song song
Trong mi trng lp trnh tun t:
Cc cu lnh c thc hin tun t.
Mi chng trnh thc hin s to ra nhng tin trnh bn
trong h thng m ngi lp trnh khng quan st c.
Mi cu lnh thc hin khng gy tr ngi cho cc cu lnh
khc trong chng trnh.
123
3.1 Gii thiu chung
Trong mi trng song song:
Cc cu lnh ca chng trnh c th thc hin an xen ln
nhau.
cng mt thi im c th c nhiu hn mt lnh c
thc hin, ngha l mi chng trnh s t ch thc hin cc
tin trnh ca mnh.
Cc chng trnh phi tng tc vi nhau v vic thc hin
ca chng nh hng ti nhp thc hin ca nhau.
Trong LTSS, ngi lp trnh khng ch vit chng trnh, d
liu nh trong mi trng tun t m cn phi s dng cc
cng c ng b ho (synchronize) v iu khin s tng
tc gia cc tin trnh.
Ngi lp trnh cn to ra v lp lch cho cc tin trnh, ngha
l s thc hin chng trnh c th nhn thy c bi ngi
lp trnh.
124
3.1 Gii thiu chung
Vn quan trng trong LTSS l phi tn dng c kh
nng ca cc b x l.
C hai cch tip cn tn dng khai thc cc b x l:
1. Pht trin nhng ngn ng lp trnh cho php th hin
c vic thc hin song song mc thut ton, v d nh
Fortran, C, v.v.
2. Xy dng nhng chng trnh dch mnh nhn
dng c cc phn on chng trnh c th thc hin
song song hay tun t.
Hai cch tip cn trn l b sung cho nhau, nu ch p dng
mt cch th khng hiu qu.
125
3.1 Gii thiu chung
Mt s phng php tip cn trong lp trnh song song:
Lp trnh song song kiu SIMD vi b nh chia s, trong
truy cp b nh l ng b (Synchronous).
Lp trnh song song kiu MIMD vi b nh chia s, trong
truy cp b nh l d b (Asynchronous).
Lp trnh song song kiu MIMD vi b nh phn tn, trong
truy cp b nh l d b.
Lp trnh song song kiu SPMD vi b nh chia s, trong
truy cp b nh l d b.
Lp trnh song song kiu SPMD vi b nh phn tn, trong
truy cp b nh l d b.
Lp trnh song song kiu MPMD vi b nh chia s, trong
truy cp b nh l d b.
126
3.2 Cc ngn ng lp trnh song song
Cc yu cu i vi mt NNLT song song
Ngoi cc yu cu i vi mt NNLT tun t, mt NNLT song song
cn phi c cc chc nng sau:
1. Cung cp cho ngi lp trnh nhng c ch khi to,
ng b v trao i gia cc tin trnh.
2. To ra c nhng chng trnh c lp vi my tnh
3. Phi h tr to ra c nhng chng trnh d c, d
vit, d chuyn i, v.v.
4. M hnh ho c vic thc hin song song.
5. C kh nng iu chnh cc tnh hung m cc tin
trnh i hi phi trao i, tng tc vi nhau.
7/17/2010
22
127
Cc tnh hung thng gp trong LT song song
1. Ti mt thi im c mt s tin trnh mun truy cp vo mt
ti nguyn chung hoc cp nht vo mt bin chia s. M
nhng ti nguyn ch cho php mt tin trnh truy cp ti
mi thi im.
2. Khi mt tin trnh c quyn truy cp vo ti nguyn chung
th n s dng ti nguyn nhng khng c ngn cn
hot ng ca nhng tin trnh khc.
3. Khi mt s tin trnh cng kt hp thc hin mt s php
ton trn c s quan st hnh ng ca nhau th ngi lp
trnh phi lp lch cho nhng tin trnh .
C hai cch gii quyt cc tnh hung trn:
128
1. Tt c cc tin trnh phi s dng cu trc d liu
chung tin trong vic cp nht v trao i vi
nhau.
2. Tt c cc tin trnh thc hin s ng b bng
cch s dng hai hm wait() v pass(). Khi trao i
vi nhau, mt tin trnh ch nhn c mt tn
hiu ca tin trnh i tc v khi nhn c th hai
tin trnh trao i trc tip vi nhau.
129
Tng qut, c hai cch pht trin NNLT song song:
1. M rng nhng ngn ng lp trnh tun t hin c,
b sung thm nhng cu trc mi thc hin
c song song v gii quyt c s xung t
trong truy cp d liu.
2. Xy dng mt ngn ng lp trnh song song mi.
130
3.2.1 V d minh ha
a. Ci t song song trn kin trc UMAvi 4 BXL:
=
n
i
n
r
m
1

n
i
n
m r
1
2
) (
trong
Bi ton: Xc nh phng sai (variance) ca mt mu c
cho bi mt danh sch cc s thc r
1
, r
2
, ...., r
n
. Ngha l phi
tnh:
BXL
1
BXL
2
BXL
4
Memory I/O Devices
Switching mechanism
. . . . . .
BXL
3
131
Ci t song song:
Cc s thc r
i
c lu tr trong b nh chia s.
Bn bin sau y cng c to ra trong b nh chia s:
Sum (lu tng gi tr):
Mean (lu gi tr trung bnh):
Sum of square (lu tng bnh phng)
Variance (lu gi tr phng sai)
Qu trnh thc hin song song c m t nh sau:
=
n
i
r S
1
=
n
i
n
r
m
1
2
1
) ( m r SS
n
i
=
n
m r
n
SS
n
i
2
1
) (
=
132
=
n
i
n
r
m
1

n
i
n
m r
1
2
) (
Sum of
Sum Mean Squares Variance
Values
36
Shared memory
0
5 4 1 5 4 2 2 3 3 1 2 4
Process 4
Temporary
Process 3
Temporary
Process 2
Temporary
Process 1
Temporary
8
7
11
10
Sum of
Values
36
Shared memory
3 0
5 4 1 5 4 2 2 3 3 1 2 4
Process 4
Temporary
Process 3
Temporary
Process 2
Temporary
Process 1
Temporary
8
7
11
10
(a): Mi tin trnh cng 3
gi tr c phn chia
v lu vo bin tm thi
ca n. Sau khi cc tin
trnh thc hin xong
vic tnh cc tng con,
s c mt tin trnh
cng dn cc gi tr ny
li v lu vo bin Sum
b nh chia s.
(b): Mt tin trnh n s
tnh gi tr trung bnh v
lu vo bin b nh
chia s Mean.
n=12; (r
i
)=(4,2,1,3,3,2,2,4,5,1,4,5)
7/17/2010
23
133
=
n
i
n
r
m
1

n
i
n
m r
1
2
) (
Sum of
Values
36
Shared memory
3 22 1.83
5 4 1 5 4 2 2 3 3 1 2 4
Process 4
Temporary
Process 3
Temporary
Process 2
Temporary
Process 1
Temporary
1
6
6
9
(c): Mi tin trnh phi xc
nh:(x
i
-Mean)
2
, (x
i
-Mean)
2
lu vo bin tm thi v
cui cng cng dn lu
vo bin ca b nh chia
s.
(d): Mt tin trnh n xc
nh variance bng cch
chia gi tr ca Sum of
Squares cho 12.
Sum of
Values
36
Shared memory
3 22
5 4 1 5 4 2 2 3 3 1 2 4
Process 4
Temporary
Process 3
Temporary
Process 2
Temporary
Process 1
Temporary
1
6
6
9
n=12; (r
i
)=(4,2,1,3,3,2,2,4,5,1,4,5)
134
3.2.1 V d minh ha (tip)
b. Ci t song song trn NUMAvi kin trc siu khi 4 BXL:
Bi ton: Xc nh phng sai (variance) ca mt mu c
cho bi mt danh sch cc s thc r
1
, r
2
, ...., r
n
. Ngha l phi
tnh:
=
n
i
n
r
m
1

n
i
n
m r
1
2
) (
trong
Ci t song song
Kin trc ny khng c b nh chia s chung; n gi tr r
1
, r
2
,
...., r
n
c phn phi cc b nh cc b ca mi BXL.
Mi nt c 4 bin: 2 bin lu tr cc gi tr cng dn l
Sum v Sum of Squares; mt bin lu gi tr trung bnh
Mean v bin cn li lu tr gi tr phng sai variance.
Qu trnh tip tc thc hin c m t nh hnh di y.
135
Nhc li,
c im ca NUMA:
C mt ng kt ni cc BXL li vi nhau.
B nh chia s c phn tn cho tt c cc BXL thnh b
nh cc b v tt c cc moun nh s l b nh chung cho
cc BXL.
Cc BXL c php truy cp ng thi ti mt hay nhiu m
un nh v c th hot ng c lp vi nhau.
Network
Cache
P
Mem Cache
P
Mem
Cache
P
Mem Cache
P
Mem
NUMA vi Distributed Memory
136
Sum1
Mean
Sum2
Variance
Values
4 2 1
7 Sum1
Mean
Sum2
Variance
Values
3 3 2
8
Sum1
Mean
Sum2
Variance
Values
2 4 5
11 Sum1
Mean
Sum2
Variance
Values
1 4 5
10
Sum1
Mean
Sum2
Variance
Values
4 2 1
36
3
Sum1
Mean
Sum2
Variance
Values
3 3 2
36
3
Sum1
Mean
Sum2
Variance
Values
2 4 5
36
3
Sum1
Mean
Sum2
Variance
Values
1 4 5
36
3
(b):Cc BXL thc hin hon i v
cng dn, sau cng chia cho 12
xc nh Mean.
(a): (b):
(a):Cc tin trnh tnh tng cc gi tr
ti mi nt.
=
n
i
n
r
m
1

n
i
n
m r
1
2
) (
n=12; (r
i
)=(4,2,1,3,3,2,2,4,5,1,4,5)
137
Sum1
Mean
Sum2
Variance
Values
4 2 1
36
3
6
Sum1
Mean
Sum2
Variance
Values
3 3 2
36
3
1
Sum1
Mean
Sum2
Variance
Values
2 4 5
36
3
6
Sum1
Mean
Sum2
Variance
Values
1 4 5
36
3
9
Sum1
Mean
Sum2
Variance
Values
4 2 1
36
3
22
1.83
Sum1
Mean
Sum2
Variance
Values
3 3 2
36
3
22
1.83
Sum1
Mean
Sum2
Variance
Values
2 4 5
36
3
22
1.83
Sum1
Mean
Sum2
Variance
Values
1 4 5
36
3
22
1.83
(c) Mi tin trnh, vi cc gi tr
ring hin c ti nt ca mnh s
thc hin tnh (x
i
-Mean)
2
v lu
vao bin Sum2.
(c): (d):
(d) Cc BXL thc hin hon i
v cng dn, sau cng chia
cho 12 xc nh Variance.
=
n
i
n
r
m
1

n
i
n
m r
1
2
) (
n=12; (r
i
)=(4,2,1,3,3,2,2,4,5,1,4,5)
138
Gii thiu mt s s ngn ng lp trnh song song
3.2.2 Fortran 90
Fortran 90 l ngn ng lp trnh chun ANSI.
Fortran 90 l ngn ng lp trnh song song theo d liu
c nng cp t Fortran 77 bng cch b sung thm mt
s c tnh nh:
Kiu d liu do User nh ngha
Php ton v Array, con tr
Cc BXL h tr vic s dng cc kiu d liu Short
Integer, Packed logical,...
Cp pht b nh ng.
7/17/2010
24
139
Ngi lp trnh Fortran 90 da vo m hnh song song tng
t nh PRAM gm c nhng thnh phn:
Mt CU (n v iu khin)
Mt n v x l logic, s hc ALU
B nh chia s.
Mt tp cc BXL
CPU thc hin cc cu lnh tun t bng cch truy cp vo
cc bin c lu tr trong b nh chia s.
n v vector lu tr, c, ghi d liu vo b nh chia s v
c CPU iu khin thc hin song song.
140
3.2 Cc ngn ng lp trnh song song - Fortran 90
Fortran 90c cc kiu d liu chun:
REAL, INTEGER, CHARACTER, LOGICAL, v.v.
Dng khai bo tng qut:
Type [(kind)] [, attribute] :: Variable-List
Trong ,
Type l kiu c s hoc kiu c nh ngha bi NSD
kind l phn tu chn cng mt s kiu c s dng
nh ngha v kiu c th
V d: CHARACTER (LEN = 10) :: s1
nh ngha bin s1 kiu CHARACTER c th cha 10 k t.
[, attribute] l danh sch cc thuc tnh ca Fortran c s
dng nh ngha cc c tnh ring ca danh sch cc
bin.
+:: phn t phn cch gia phn m t kiu v danh sch cc
bin.
Variable-List: danh sch cc bin
141
V d v khai bo v mng:
INTEGER, DIMENSION (1 : 10) :: SA
{Bin mng SA c 10 phn t kiu s nguyn.}
+ Khai bo mt hng mng:
(/ 2,4,6,8,10 /) hoc (/ (I, I = 2, 10, 2) /)
+ Fortran 90 cho php p dng cc php ton s hc cho cc
bin mng, ngha l ton hng ca cc php ton s hc (+,
-, *, /) c th l bin n hoc bin mng.
V d v php ton trn bin mng:
INTEGER A(10), B(10), C
DO I = 1, 10, 1
A(I) = B(I) + C
END DO
Ta c th vit ngn gn hn.
A = B + C
142
Cu trc chng trnh Fortran
PROGRAM <tn chng trnh>
IMPLICIT NONE
[phn c t]
[phn thc hin]
[phn chng trnh con]
END PROGRAM <tn chng trnh>
143
Xy dng mt chng trnh FORTRAN
144
V d: tnh s t t cng thc bng NNLT Fortran 90
1. INTEGER, PARAMETER :: N = 131000
2. INTEGER, PARAMETER :: LONG =SELECTED_REAL_KIND(13,99)
3. REAL(KIND=LONG) PI, WIDTH
4. INTEGER, DIMENSION(N)::id
5. REAL(KIND=LONG), DIMENSION(N)::X, Y
6. WIDTH = 1.0_LONG/N
7. ID = (/(I,I = 1, N)/)
8. X = (ID 0.5)* WIDTH
9. Y = 4.0 / (1.0 + X * X)
10.PI = SUM(Y) * WIDTH
11.10 FORMAT(PI = , F14.12)
12.PRINT 10, PI
13.END
.
khai bo tham s N l
s cc on con ti a
tnh tch phn
khai bo tham s LONG s
dng cho cc bin s thc vi
phn tnh c t nht 13 ch s
v phn ng (m) thuc
khong [10
-99
,10
99
]. Cc bin
thc ny c khai bo dng
3 v 5
xc nh rng ca mi
khong con. Bin mng ID gm
N phn t khai bo dng 4,
c khi to dng 7
tnh ton song song im gia
ca mi khong con
tnh ton song song gi tr hm
tng ng ti cc im gia X
gi hm tnh tng SUM
Khai bo bin mng id c
N phn t kiu s nguyn
7/17/2010
25
145
Tiu lun: Fortran 90 Tutorial (02 hc vin)
Professor Dr.Shene
Department of Computer Science
Michigan Technological University
www.cs.mtu.edu/~shene/COURSES/cs201/NOTES/fortran.html
http://www.fortran-2000.com/
146
3.2 Cc ngn ng lp trnh song song - OCCAM
3.2.3 Lp trnh song song vi OCCAM
Gii thiu v OCCAM:
NNLT song song c Inmos Company (Anh) pht trin
nm 1988,
Mc ch chnh l thit k v ci t cc chip c gi l
transputer.
Chng trnh Occam thng nhiu tin trnh v chng c
th c nh x sang mt s cc transputer bt k thc
hin song song v trao i d liu vi nhau thng qua cc
knh vo/ra.
Occam l NNLT bc cao, c s dng lp trnh cho
nhng h thng gm nhiu my tnh kt ni vi nhau, hoc
cc h phn tn.
Trong Occam, cc hnh ng c th thc hin song song
c gi l tin trnh v mi cu lnh cn phi khai bo nh
mt tin trnh.
147
M phng m hnh lp trnh trong Occam
A: Cc tin trnh thc
hin tun t
B: Cc tin trnh thc
hin song song
C: Cc tin trnh thc
hin tun t
D: Cc tin trnh thc
hin song song
E: Cc tin trnh thc
hin tun t
A
B
C
D
E
148
3.2 Cc ngn ng lp trnh song song - Occam
Cc tin trnh trong Occam
C ba tin trnh nguyn thu trong Occam:
Tin trnh gn: thay i gi tr ca cc bin
Tin trnh Input: nhn d liu vo t cc knh vo (cng vo)
Tin trnh Output: gi d liu ra cc knh ra.
Cu trc ngn ng
Tin trnh gn:
sum := partion1 + partion2
/*gn gtr cho bin sum l tng hai gtr partion1v partion2 */
Tin trnh Input: user1 ? x
/*gn gi tr t knh user1 cho x */
Tin trnh Output: C ! x * x
/*gi gi tr x
2
ra knh C */
149
Cc cu trc iu khin
SEQ: cu trc tun t, cc thnh phn trong tin trnh ny
c thc hin theo th t v tin trnh ny kt thc khi
thnh phn cui cng thc hin xong. Mi thnh phn ca
tin trnh ny c th l mt tin trnh song song.
V d:
SEQ
channel1 ? partial1
channel2 ? partial2
sum:= partial1 + partial2
partial3 ! Sum
Cc s nguyn c nhp
vo tun t t 2 knh khc
nhau sau cng li v
chuyn sang knh 3
150
Cc cu trc iu khin (cont.)
PAR: cu trc song song, cc thnh phn ca tin trnh ny
c thc hin ng thi v tin trnh ny s kt thc khi tt
c cc thnh phn ca n u kt thc.
V d:
SEQ
PAR
channel1 ? partial1
channel2 ? partial2
sum:= partial1 + partial2
Cc s nguyn c
nhp vo song song t 2
knh khc nhau sau
cng li
7/17/2010
26
151
Cc cu trc iu khin (cont.)
ALT: cu trc tuyn chn, chn mt trong cc thnh phn ca
tin trnh thc hin nu n tho mn iu kin la chn
v tin trnh ny kt thc khi thnh phn c la chn kt
thc. (xem v d tnh tch phn)
152
Cc cu trc khc
1. Cu trc iu khin IF, gi l tin trnh iu kin chn mt
tin trnh thnh phn khi biu thc Boolean c gi tr true;
2. Cu trc lp WHILE, gi l tin trnh lp thc hin lp li
tin trnh thnh phn cho n khi biu thc iu kin
Boolean nhn gi tr true.
3. Cu trc lp FOR, gi l tin trnh lp c s vng lp xc
nh trc, i = [0 FOR n]
V d: PAR i = [0 FOR 9]: to ra 10 tin trnh song song, i
nhn gi tr t 0 n 9
153
V d: Gi s c hai tin trnh ging nhau cng nhn d liu vo v
cng dn vo tng.
CHAN In1, In2:
CHAN Out1, Out2:
VAR Sum1, Sum2:
SEQ &cc lnh di y thc hin tun t
Sum1 := 0 &gn gi tr u cho Sum1 v Sum2
Sum2 := 0
PAR &bt u thc hin song song
-- Tin trnh th nht
While TRUE
VAR Item1:
SEQ
In1 ? Item1 &gn gi tr knh In1 cho Item1
Sum1 := Sum1 + Item1
Out1 ! Item1
-- Tin trnh th hai
While TRUE
VAR Item1:
SEQ
In2 ? Item2
Sum2 := Sum2 + Item2
Out2 ! Item2
154
S trao i gia cc tin trnh
Trong v d trn, cc tin trnh khng cn trao i vi nhau v
mi tin trnh u s dng cc bin cc b ca ring n.
Tuy nhin, khi c nhiu tin trnh mun trao i d liu vi
nhau th phi trao i trn cng mt knh truyn d liu.
Ngha l, mt tin trnh gi d liu ra mt knh truyn v
tin trnh kia nhn d liu t knh .
Trong Occam, mi tin trnh thc hin trn mt b x l v
truyn thng ip (Passed-Message) ti nhng b x l ln
cn theo kin trc hnh khi.
155
1. DEF N = 400000:
2. DEF PROCESS = 8:
3. DEF CHUNK = N / PROCESS:
4. CHAN sum[PROCESS]
5. PAR
6. PAR i = [0 FOR PROCESS]
7. REAL64x, localsum, width:
8. SEQ
9. localsum := 0.0
10. width := 1.0 / N
11. x := ((i * CHUNK) + 0.5) * width
12. SEQ i = [0 FOR CHUNK]
13. SEQ
14. localsum:=localsum+(4.0/(1.0+(x*x))))
15. x := x + width
V d: tnh s t t cng thc bng NNLT Occam
cont.
T dng 1-3:
nh ngha cc hng:
N l s khong chia;
PROCESSES l s tin
trnh s c to;
CHUNK l s khong
cho mi tin trnh (
chng trnh thc hin
chnh xc nn chn N
l bi s ca
PROCESSES
Dng 4: nh ngha mt
knh cho mi tin trnh.
Tin trnh s s dng knh
ny gi d liu ca n
(subtotal) ra tin trnh bn
ngoi (grand total)
T dng 5-31: l nhng tin
trnh song song v tun t,
gm c (PROCESS+1) tin
trnh. Mi tin trnh trong cc
tin trnh thc hin song song
t dng 6-17, s xc nh tng
din tch cc hnh ch nht
c chia s
156
16. localsum := localsum * width
17. sum[i] ! localsum
18. REAL64 pi :
19. INT got [PROCESS] :
20. SEQ
21. pi := 0.0
22. SEQ i = [0 FOR PROCESS]
23. got[i] := FALSE
24. SEQ i = [0 FOR PROCESS]
25. REAL64 y:
26. SEQ
27. ALT i = [0 FOR PROCESS]
28. (got[i] = FALSE) & sum[i] ? Y
29. got[i] := TRUE
30. pi := pi + y
31. Output ! Approximation to pi is: ; pi
V d: tnh s t t cng thc bng NNLT Occam
T dng 18-31: l tin
trnh cui cng, xc
nh tng sau cng v
chuyn cho tin trnh
output in kt qu pi
7/17/2010
27
157
Tiu lun (02 hc vin):
OCCAM Language
http://en.wikipedia.org/wiki/Occam_programming_language
http://www.wotug.org/
Tiu lun (02 hc vin): ThreadMentor
http://cs.mtu.edu/~shene/NSF-3/e-Book/index.html
158
3.2.4 Lp trnh song song vi PVM (Parallel Virtual Machine)
PVM c pht trin bi University of Tennessee, Oak Ridge
National Laboratory and Emory University (1989)
PVM l mt phn mm s dng cho h thng bao gm cc
my tnh khng thun nht (heterogeneous) c kt ni vi
nhau x l song song.
PVM thng c s dng cho nhng my tnh ni mng
trong mi trng UNIX hoc Windows
3.2 Cc ngn ng lp trnh song song - PVM
159
Cc c im chnh ca PVM:
Thc hin theo m hnh truyn thng ip (Message Passing)
H tr s kt ni khng thun nht (Heterogeneity): PVM h tr
s kt ni ca nhiu my tnh, nhiu mng my tnh v nhiu
loi chng trnh ng dng khc nhau.
H tr a b x l: PVM s dng nhng kh nng truyn thng
ip trong h a b x l khai thc ht kh nng ca phn
cng.
Tnh ton da trn tin trnh: n v iu khin thc hin song
song trong PVM l mt tc v, l mt lung (thread) lm
nhim v iu khin s truyn thng v tnh ton.
Thay i cu hnh theo yu cu: Cc chng trnh c th thc
hin trn tp cc my c la chn theo yu cu ca NSD.
160
PVM x l tt c cc vn :
nh tuyn truyn thng ip
chuyn i d liu
lp lch trong mng my tnh
H thng PVM gm hai thnh phn chnh:
1. Khi pvmd (hoc pvm3) t thng tr trn tt c cc my
tnh to ra my o (cc BXL gi lp).
2. Th vin cc chng trnh con giao din ca pvm: cha cc
chng trnh con truyn thng ip, qun l cc tin trnh,
phi hp cc tc v v thay i cc my o.
161
M hnh tnh ton ca PVM
Nhp d liu v phn on
Xut d liu v hin th
kt qu
My tnh 1
My tnh 2
162
Mt kin trc ca PVM
MPP
Bridge
SIMD
Cluster 1
Cluster 2
Cluster 3
7/17/2010
28
163
Phng thc thc hin chng trnh trong PVM:
Nhng chng trnh vit bng C/C
++
, Fortran 77 c th cha
nhng li gi cc hm th vin ca PVM (y l nhng ngn
ng lp trnh c PVM h tr)
Cc chng trnh c dch theo kin trc ca h thng
(host pool), cc tp m ch (object file) c t vo nhng
ni m mi my tnh ca h thng u truy cp c.
Cc USER to ra bn sao ca tc v ch (master) hoc khi
ng mt tc v thc hin cc ng dng ca mnh.
Mt tin trnh c khi ng bi mt tin trnh khc c
gi l tin trnh t (slave).
Nhng tin trnh ny c th thc hin mt s tnh ton cc b
v trao i vi nhau gii quyt bi ton t ra.
164
Mt vi thng tin v PVM:
User mun chy mt ng dng trn PVM th phi khi ng
PVM v to ra mt my o.
Mt ng dng PVM c th c gi t du nhc ca h iu
hnh UNIX mt host no
Cc Users c th cu hnh my o v thc hin cc ng
dng ca mnh vi PMV
Trong PVM mi tin trnh u c th giao tip v/hoc ng
b vi nhng tin trnh khc
Cc files PVM c th truy cp a ch:
http://www.netlib.org/pvm3/index.html
Nhiu my tnh h tr h thng PVM thc hin nh: BBN
Butterfly TC2000, 80386/486 PC chy vi UNIX, Thinking
Machines CM-2, CM-5, Cray-2, Cray-5MP, HP-9000 PA-
RISC, Intel Paragon, Silicon Graphics IRIS, Sun 4, DEC Micro
VAX, v.v.
165
Gii thiu mt chng trnh trn PVM
Chng trnh in ra mn hnh (master) nh danh ca tc v (ID task)
nhn c t hm pvm_mytid(),
Sau s dng cc hm:
pvm_spawn(): to ra mt bn sao v gi chng trnh hello_other
pvm_recv(): nhn v thc hin khi thng ip gi n.
pvm_exit(): kt thc chng trnh trong h thng PVM.
166
Gii thiu mt chng trnh trn PVM (cont.)
Chng trnh slave:
pvm_parent() xc nh ID task ca master.
Sau dng cc hm di y xc nh hostname v chuyn
hotname ny cho master:
pvm_initsend(): khi to buffer gi
pvm_pkstr(): t mt xu vo buffer gi i
pvm_upkstr(): c mt xu vo buffer
pvm_send(): chuyn d liu buffer ti tin trnh nhn c xc
nh bi pvm_mptid.
167
/* Chng trnh master c tn Hello.c */
#include pvm3.h
main(){
int cc, tid, msg;
char buf[100];
printf(Master ID number %x\n, pvm_mytid());
cc= pvm_spawn(Hello_other,(char**)0,0,, 1, &tid);
if(cc== 1){
msg = 1;
pvm_recv(tid, msg);
pvm_upkstr(buf);
printf(From master %x: %s\n, tid, buf);
} else printf(Cannot start hello_other\n);
pvm_exit();
}
to ra mt bn sao v gi
chng trnh hello_other
nhn v thc hin khi
thng ip gi n
c mt xu vo buffer
kt thc chng trnh
trong h thng PVM 168
/* Chng trnh slave c tn: hello_other.c */
#include pvm3.h
main(){
int ptid, msg;
char buf[100];
ptid = pvm_parent();
strcpy(buf, Hello world from );
gethostname(buff + strlen(buf), 64);
msg = 1;
pvm_initsend(PvmDataDefault);
pvm_pkstr(buf);
pvm_send(ptid, msg);
pvm_exit();
}
pvm_parent() nhn mt tc v
ca tin trnh t t tin trnh ch
pvm_initsend() khi
to buffer gi
pvm_pkstr() t mt xu
vo buffer gi i
pvm_send() chuyn d liu buffer ti
tin trnh nhn c xc nh bi ptid.
7/17/2010
29
169
Tiu lun
1. Tm hiu v PVM
http://www.csm.ornl.gov/pvm/intro.html
http://www.csm.ornl.gov/pvm/
http://search.cpan.org/dist/Parallel-Pvm/
http://www.parallels.com/
http://www.netlib.org/pvm3/book/node1.html
170
Tiu lun
2. Thread trong JAVA
http://java.sun.com/j2se/1.4.2/docs/api/java/lang/Thr
ead.html
http://en.wikipedia.org/wiki/Multithreading
Mt s v d v cch to thread trong JAVA
http://www.niitvn.com/niitwcms/dt/QLT/MotcachtaoT
hreadtrongJava.pdf
http://www.niit-n.com/4rum/showthread.php?p=8754
Introduction to Java threads:
http://www.javaworld.com/javaworld/jw-04-1996/jw-
04-threads.html
171
Mc ch:
Xc nh mc ph thuc d liu v quan h u tin tnh
ton gia cc tin trnh
V d: Xt dy cc lnh sau:
S
1
: A := B + C
S
2
: B := A + E
S
3
: A := A + B
3.3 S ph thuc d liu v th u tin ca cc tin trnh
S
1
S
2
S
3
d
1
d
2
d
3
d
4
d
5
d
6
th ph thuc d liu
d
7
Lnh S
1
tnh gi tr ca bin
A v bin ny c s dng
trong S
2
v S
3
. Do vy, c s
ph thuc ca S
2
, S
3
vo S
1
(k hiu l d
1
, d
2
).
Lnh S2 tnh gi tr ca bin
B v bin ny c s dng
trong S3. Do vy, c s ph
thuc ca S3 vo S2 (k
hiu l d3).
Gi tr trc ca bin B
c s dng S1. Do vy,
c s ph thuc d4.
C hai lnh S1 v S3 cng
tnh gi tr ca bin A v do
vy, c s ph thuc d5.
Lnh S3 tnh gi tr ca bin A v
bin ny c s dng trong S2 v
S3. Do vy, c s ph thuc ca
S2, S3 vo S3 (k hiu l d6, d7).
Nhn xt: Cc ph thuc ny
chng ging nhau t no c!!!
172
th ph thuc d liu?
l mt th c hng G=(V,E), trong V l tp cc lnh
trong chng trnh, E l cc ph thuc d liu.
V d
d
7
S
1
S
2
S
3
d
1
d
4
d
2
d
5
d
3
d
6
S
1
: A := B + C S
2
: B := A + E S
3
: A := A + B
173
K hiu:
DEF(S) - tp tt c cc bin c gi tr b thay i khi thc hin
cu lnh S.
USE(S) - tp tt c cc bin c truy cp (c s dng)
khi thc hin cu lnh S.
V d: S
1
: A := B + C S
2
: B := A + E S
3
: A := A + B
DEF(S
1
) = {A}
DEF(S
2
) = {B}
DEF(S
3
) = {A}
USE(S
1
) = {B,C}
USE(S
2
) = {A,E}
USE(S
3
) = {A,B}
174
Phn loi cc ph thuc d liu:
Xt cc cu lnh S
1
: A := B + C; S
2
: B := A + E ; S
3
: A := A + B
1. Ph thuc dng d liu (Data Flow Dependency):
l s ph thuc d liu gia S
1
v S
2
khi DEF(S
1
)USE(S
2
) |
y l loi ph thuc rt chung v rt kh loi b bi v lnh S
2
yu cu gi tr ca mt bin v gi tr ny phi c tnh S
1
.
Khi xut hin ph thuc dng d liu gia cc cu lnh th
chng khng thc hin song song c.
Quan h ph thuc dng d liu c k hiu
V d: cc ph thuc d
1
, d
2
, d
3
l loi ph thuc dng d liu.
S
1
S
2
S
3
d
1
d
2
d
3
Ph thuc dng d liu
7/17/2010
30
175
2. Phn ph thuc d liu (Data Anti-Dependency):
1
v S
2
khi DEF(S
2
)USE(S
1
) |
S ph thuc ny xut hin khi chng ta s dng li tn gi
ca cc bin, mt bin c s dng trong S
1
v sau
c nh ngha li S
2
.
Khi xut hin phn ph thuc d liu gia cc cu lnh th
chng cng khng thc hin song song c.
Quan h phn ph thuc d liu c k hiu:
V d: cc ph thuc d
4
, d
6
. d
7
l cc phn ph thuc d liu.
S
1
S
2
S
3
d
4
d
6
Phn ph thuc d liu
d
7
S
1
: A := B + C; S
2
: B := A + E ; S
3
: A := A + B
176
3. Ph thuc d liu ra (Data Output Dependency):
1
v S
2
khi DEF(S
2
)DEF(S
1
) |
S ph thuc ny xut hin do hai nguyn nhn:
S dng li tn ca cc bin (dng chung)
Tnh tng gi tr ca cng mt bin.
Nu nhng lnh ny thc hin ng thi th chng s ghi
cc gi tr vo cng mt nh. Do vy, cn phi xc nh
chnh xc th t thc hin ngn nga vic s dng
nhng gi tr khng ng.
Quan h ph thuc d liu ra c k hiu l:
V d: ph thuc d
5
l loi ph thuc d liu ra.
S
1
S
3
d
5
Ph thuc d liu ra
S
1
: A := B + C;
S
3
: A := A + B
DEF(S
1
)DEF(S
3
) ={A} |
177
4. Ph thuc d liu vo (Data Input Dependency):
1
v S
2
khi USE(S
2
)USE(S
1
) |
Bi v cc lnh ny ch truy cp (c) v khng lm thay i gi
tr ca cc bin , do vy cc lnh ny c th thc hin theo
bt k th t no cng c, ngha l c th thc hin song
song.
Quan h ph thuc d liu vo c k hiu:
V d
S
2
: B := A + E ; S
3
: A := A + B
USE(S
3
) USE(S
2
) ={A} |
S
2 S
3
Ph thuc d liu vo
d
3
178
5. Ph thuc iu khin d liu (Data Control Dependency):
1
v S
2
khi s thc hin ca
lnh ny ph thuc vo gi tr ca cc bin c tnh lnh
kia.
Quan h ph thuc iu khin d liu
c k hiu l:
S
4
: P = (B => 0)
S
5
: if (P is true) then
D = 1
else D = 2
endif
S
4 S
5
Ph thuc iu khin d liu
179
Nhn xt:
tng c mc song song ca chng trnh chng ta c
th kh hoc gim cc phn ph thuc d liu v ph thuc d
liu ra trong th ph thuc d liu.
Thng thng kh cc ph thuc d liu ny l t li tn
cho cc bin trnh vic chia s cc bin .
V d: Xt on chng trnh sau:
for i = 1, n, 1
X = A[i] + B[i]
Y[i] = 2 * X
X = C[i] * D[i]
P = X + 15
endfor
for i = 1, n, 1
X = A[i] + B[i]
Y[i] = 2 * X
XX = C[i] * D[i]
P = XX + 15
endfor
Nhn xt:
C phn ph thuc d liu. Nu hai lnh sau ta t X thnh XX th s to
ra hai on chng trnh c th thc hin song song:
180
+ S dng cc bin khc nhau loi b ph thuc d liu ra.
V d:
A = B + C
A = C * X
A = B + C
A1 = C * X
Nhn xt:
Hai lnh trn c s ph thuc d liu ra.
C th loi b ph thuc ny bng cch
thay bin A ca cu lnh th hai bng A
1
DEF(S
2
) DEF(S
1
) = {A} |
7/17/2010
31
181
+ S dng cc bin khc nhau loi b phn ph thuc d
liu
V d:
Nhn xt:
Hai lnh trn c quan h phn ph thuc d liu.
C th loi b s ph thuc ny bng cch s
dng mt bin khc cho cu lnh th hai
A = B + C
B = C * X
A = B + C
B1 = C * X
DEF(S
2
)USE(S
1
) = {B} |
182
+ Thc hin mt s php bin i (php th) cng c th loi
b c phn ph thuc d liu.
V d: Xt dy cc cu lnh sau:
S
1
: A = B + C
S
2
: B = A * 3
S
3
: A = 2 * C
S
4
: P = (B => 0)
if (P is true) then
S
5
: D = 1
S
6
: else D = 2
endif
S
1
S
3
S
2
S
4
S
5
S
6
th ph thuc d liu
Nhn xt: x l song song, th cn thit phi loi b i mt s loi ph
thuc d liu c th.
183
V d: loi b nhng quan h phn ph thuc d liu v ph
thuc d liu kt qu
S
2
: B = (B + C) * 3
S
3
: A = 2 * C
S
4
: P = (B => 0)
if (P is true) then
S
5
: D = 1
S
6
: else D = 2
endif
S
1
: A = B + C
S
2
: B = A * 3
S
3
: A = 2 * C
S
4
: P = (B => 0)
if (P is true) then
S
5
: D = 1
S
6
: else D = 2
endif
S
3
S
2
S
4
S
5
S
6
th ph thuc d liu rt gn
S
1
S
3
S
2
S
4
S
5
S
6
DEF(S
2
)USE(S
1
) ={B} |
(phn ph thuc d liu)
184
for (i = 0; i < N; i++)
A = A + i;
Nhn xt:
Trong mi bc lp u phi s dng nhng tn bin khc nhau nu mun
loi b phn ph thuc d liu. iu ny kh thc hin c trong trng
hp s vng lp ln hoc bt k.
Quan h ph thuc d liu l khng c tnh bc cu. Ngha l nu S
2
ph
thuc vo S
1
, S
3
ph thuc vo S
2
th khng kt lun c S
3
ph thuc d
liu vo S
1
.
Tuy nhin, nhng php bin i n gin nh th khng phi
lc no cng p dng thnh cng. V d, xt chu trnh sau:
i = 0;
aa: if(i > N) stop;
a = a + i;
i++;
goto aa;
khng s dng FOR
185
DO i
A[2*i+1] = B[i] (1)
D[i] = A[2*i] (2)
ENDDO
Nu xem mng A nh mt thc th n th hai lnh trn l ph
thuc theo dng d liu (v A xut hin trong DEF ca (1) v
cng xut hin trong USE ca (2)).
Nhng nu ta xt thm ch s mng th trong cu lnh (1) s
dng cc phn t mng A c ch s l; cn (2) li s dng
phn t mng A c ch s chn.
Do , y khng c s ph thuc d liu.
Ph thuc theo chu trnh v theo mng
Mc ch: kh ph thuc d liu trong cc vng lp.
Xt on chng trnh sau:
hai cu lnh (1) v (2)
c thc hin song song
c khng? ?
186
Xt v d sau:
FOR I=1 DO
A[I] = A[I-1] + 1 (3)
Chu trnh trn c th khai trin thnh:
A[1] = A[0] + 1; A[2] = A[1] + 1;
A[3] = A[2] + 1; A[4] = A[3] + 1; . . .
Vn : khi xy dng cc chu trnh lp phi xt xem nhng dy
lp khc nhau ca chng c th thc hin song song c
khng?
Ni cch khc, khi mt b x l P thc hin mt dy cc cu
lnh vi i = 1 (bin iu khin vng lp) th b x l Q c th cng
thc hin dy cc cu lnh vi i=2 hay khng?
7/17/2010
32
187
Xt hai vng lp th i v th j tng ng vi I = i v I = j.
Khng mt tnh tng qut ta c th gi thit i < j. Chu trnh s c
dng:
. . .
A[i] = A[i-1] + 1;
. . .
A[j] = A[j-1] + 1;
. . .
Nhn xt:
Cc tp trn khng cha bin n m ch c cc bin c ch s
v khi i v j khc nhau th cc bin khc nhau.
Hai tp DEF(i)= {A[i]} v USE(j) = {A[j-1]} c cha phn t
chung khi i = j 1, V phng trnh i = j 1 c v s nghim
Vy kt lun chu trnh trn khng thc hin song song c.
USE(i) = {A[i-1]}
USE(j) = {A[j-1]}
DEF(i) = {A[i]}
DEF(j) = {A[j]}
188
Nhn xt:
Ni chung, vic xc nh s ph thuc theo cch nh trn l
bi ton kh. S phc tp s xut hin bi v c nhiu dng
phng trnh khng hin nhin. Nhng phng trnh ny khng
c li gii trc tip trong cc cng c ton hc kinh in.
Trng hp tng qut, khi xt n vng lp m dn n dng
phng trnh a
1
x
1
+ a
2
x
2
+ + a
n
x
n
= c vi a
i
v c l cc s
nguyn (gi l phng trnh Diophantine) th kt lun chu trnh
khng th thc hin song song.
Lu : Phng trnh Diophantine c dng tng qut:
a
1
x
1
+ a
2
x
2
+ + a
n
x
n
= c vi a
i
v c l cc s nguyn.
Nm 1991 Zima v Chapman chng minh c rng nu
c s chung ln nht (a
1
,a
2
, a
n
) chia ht cho c th phng
trnh Diophantine c mt nghim nguyn.
Trng hp tng qut, gii phng trnh trn l khng th.
189
3.4 Bin i chng trnh
Mc ch:
1. Loi b c s ph thuc ca cc cu lnh trong ch.trnh.
2. Khc phc c nhng tr ngi ca cch xc nh cc vng
lp c th thc hin song song dn ti phng trnh Diophantine.
3.4.1 Cc bin qui np
Bin qui np l loi bin trong chu trnh m cc gi tr lin tip
ca n to ra dy cp s cng.
V d: xt on chng trnh sau:
m = 0;
DO i = 1 to N
m = m + k;
x[m] = a[i]
END
{m l bin qui np}
190
3.4 Bin i chng trnh
Nhn xt:
Hai cu lnh trn c s ph thuc d liu v m c s dng
cho gia cc vng lp.
Khi vng lp trc tnh gi tr ca m th m li c s dng
lnh th hai nn chng phi thc hin tun t.
Nu mi vng lp chng ta d bo c gi tr ca m th c
th s loi b c s ph thuc d liu lin quan n m.
Gi tr k l bt bin trong cc vng lp v c cng lin tip
vo sau mi vng lp, do vy, vng lp th i, m = i * k.
m = 0;
Do i = 1 to N
m = m + k;
x[m] = a[i]
End
Do i = 1 to N
x[i*k] = a[i]
End
191
3.4 Bin i chng trnh
3.4.2 S ph thuc tin
Ph thuc tin l loi phn ph thuc d liu gia cc vng
lp ca chu trnh.
Do i = 1 to N
x[i] = x[i+1]
End
Nhn xt:
Chu trnh khng thc hin song song c
Chu trnh lm nhim v sao tt c cc phn t ca mt mng
sang chnh mng vi ch s gim i mt, do vy n phi thc
hin tun t.
Cch gii quyt
S dng 2 mng: mt bn gc v mt bn sao ca mng gc
vit li chng trnh
Do i = 1 to N
x1[i] = x[i]
End
Do i = 1 to N
x[i] = x1[i+1]
End
C th thc hin song song ?
C th
192
3.4 Bin i chng trnh
3.4.3 S ph thuc li
Trng hp cc phn t ca mng trong chu trnh li c
xc nh thng qua nhng phn t ng trc chng (li v
pha trc)
Do i = 1 to N
x[i] = x[i-1] + y[i]
End
Nhn xt:
Vic loi b nhng ph thuc d liu kiu ny khng n gin
Khng c gii php chung loi b ph thuc d liu trong
trng tng qut. Nu c gii php loi b th gi phi tr cng
kh t.
Hin nhin khng cn s ph thuc li trong chu trnh, tuy nhin
thi gian tnh ton trong mi cu lnh li tng ln kh nhiu.
Do i = 1 to N
x[i] = x[0]+y[1]+y[2]+ +y[i]
End
7/17/2010
33
193
3.4 Bin i chng trnh
3.4.4 S phn tch chu trnh (ph thuc li)
Tch mt chu trnh thnh nhiu chu trnh con thc hin song
song
Do i = 1 to N
a[i] = b[i] + c[i];
c[i] = a[i-1]
End
Nhn xt:
S tham chiu gia cc mng a v c dn n s ph thuc
gia cc vng lp ca chu trnh.
Gi tr c[i] lnh th nht lun l nhng gi tr lu tr c ca
mng c (c xc nh lnh th 2)
a[i-1] c s dng lnh th hai l gi tr c tnh
vng lp trc (vng th i -1)
Do i = 1 to N
a[i] = b[i] + c[i];
End
Do i = 1 to N
c[i] = a[i-1]
End
194
3.4 Bin i chng trnh
3.4.4 S phn tch chu trnh (ph thuc tin)
Do i = 1 to N
a[i] = b[i] + c[i];
c[i] = a[i+1]
End
Nhn xt:
Phn t c[i] lnh hai c tnh theo gi tr ca a[i+1], l gi
tr trc khi n c cp nht lnh th nht
Ngc li c[i] c s dng lnh th nht li l gi tr c
trc khi c cp nht lnh th hai.
Do i = 1 to N
x[i] = c[i];
c[i] = a[i+1]
End
Do i = 1 to N
a[i] = b[i] + x[i];
End
Cch gii quyt
Sao mng c sang mng ph x. Kt qu bin i chu trnh ny
thnh hai chu trnh trong khng cn s ph thuc d liu
195
3.4 Bin i chng trnh
3.3.5 Cc chu trnh lng nhau
Xt trng hp nhiu chu trnh lng nhau (chu trnh nhiu ch s)
Do i = 1 to N
Do j = 1 to N
x[i,j] = x[i, j-1] + z[i];
End
Nhn xt:
Mng z c s dng trong chu trnh hon ton khng gy ra
s ph thuc no c.
Ch s th nht ca mng x l i, cng chnh l ch s ca chu
trnh, nhng khng c s tham chiu cho gia cc vng lp.
Vy, chu trnh lp bn trong (chu trnh lp theo i) c th thc
hin song song.
Ch s th hai ca mng x l j, l ch s ca chu trnh th nht
do c s ph thuc dng d liu ca cc ln lp vng
ngoi. Cng nh 3.4.3, trng hp ny khng c cch bin
i n gin v dng kh song song.
196
Bi tp
3.1 Xc nh s ph thuc d liu ca cc lnh trong chu trnh
sau:
Do i = 1 to N
e[i] = x[i] - z[i];
a[i+1] = e[i] + 2*d[i]
a[i] = e[i]
end
3.2 Hai chu trnh sau c tng ng v ni dung tnh ton hay
khng? hy bnh lun v kh nng thc hin song song ca
cc chu trnh .
1. Do i = 1 to N
a[i] = a[i+1] + i
end
2. Do i = N downto 1
a[i] = a[i+1] + i
end
197
Bi tp
3.3 Xc nh tt c cc s ph thuc d liu trong on
chng trnh sau:
int a[MAX];
read(a);
for(i = 0; i < N; i++)
for(j = 0; j < i; j++){
a[i] = max(a[i], a[j]);
a[j] = min(a[i], a[j]);
}
3.4 Loi b cc ph thuc d liu ra v phn ph thuc d liu
ca chu trnh sau
for(i = 0; i < N; i++){
x = a[i]+ b[i]);
y[i] = 2 * x;
}
198
Bi tp
3.5 Phn tch on chng trnh sau, xc nh cc ph thuc d
liu v v th ph thuc d liu ca on chng trnh .
A = B + C;
for(i = 0; i < N; i++){
D[i] = A + b[i]);
S = b[i] * 5;
T = T + S;
}
3.6 Vit chng trnh gii phng trnh bc hai v v th ph
thuc d liu ca n.
7/17/2010
34
199
CHNG 4. CC M HNH V LP TRNH SONG SONG
NI DUNG
1. Lp trnh b nh chia s
2. Lp trnh song song da vo cc tin trnh
3. Lp trnh song song da vo cc lung
4. Lp trnh theo m hnh truyn thng ip.
5. Lp trnh trn cm my tnh vi PVM
200
1. LP TRNH B NH CHIA S
Mt vi ch (1/4)
1. H thng a b x l i xng SMP (symmetric multiprocessor Sysstem)
Cc b x l l nh nhau
Khng c nhng BXL c bit x l vo/ra
Khng c BXL c gn nhim v c bit no khc.
2. nghin cu v XLSS, chng ta khng nht thit phi c h
a b x l vt l.
3. Trong mi trng UNIX, chng ta c th to ra nhiu tin trnh
khc nhau trong h thng v chng c s dng m
phng lp trnh a b x l.
4. Hu ht cc h UNIX u cho php to ra mt s cc tin trnh
bt k v chng c lp lch cho nhng b x l thch hp
201
Mt vi ch (2/4)
5. V nguyn tc, chng trnh l tp cc tin trnh v vic vit
chng trnh l c lp vi cc b x l, vic phn chia cc
tin trnh cho cc b x l l cng vic ca h iu hnh..
6. Trong nhng h thng a nhim nh UNIX, s b x l v s
cc tin trnh khng nht thit phi bng nhau. Nhng nn t
chc mt tin trnh c mt BXL m nhim tin vic lp lch.
7. Mt vn quan trng cn xem xt khi XLSS l cn bao nhiu
nt (BXL) chng trnh song song thc hin hiu qu nht?
8. Nhiu chng trnh c xy dng ph thuc vo mt cu
hnh xc nh. Ni chung, loi chng trnh ph thuc nh vy
l khng tt, v khi b x l bn th cc tin trnh tip theo phi
ch.
202
Mt vi ch (3/4)
Trong mi trng lp trnh chia s b nh c hai rng buc
quan trng:
1. Mt tin trnh c th ch mt khong thi gian bt k gia hai
lnh cn thc hin. Gi s b x l P thc hin mt chng
trnh c mt 100 lnh, b x l Q thc hin chng trnh c 10
lnh v cng bt u thc hin ng thi. Thm ch, tt c cc
lnh c tc thc hin nh nhau cng khng th ni rng Q
s kt thc trc P.
2. Khng th xem cc lnh thc hin l nguyn t mc cc
ngn ng lp trnh. V d, mt lnh n gin nh: a = a + 1 s
l mt dy bn lnh trong ngn ng my. M ta cng bit rng,
cc tin trnh v h iu hnh ch nhn bit c cc cu lnh
ca ngn ng my.
203
Mt vi ch (3/4)
Trong lp trnh b nh chia s:
Cc tc v (tasks) s c/ghi d liu t b nh chia s qua
khng gian a ch b nh chung (common address space)
C nhng c ch khc nhau nh (locks/semaphores) iu
khin vic truy nhp n b nh chia s
Ngi lp trnh khng cn m t vic truyn thng d liu, do
vic vit chng trnh s n gin.
Kh qun l d liu v khng can thip trc tip qu trnh truyn
d liu.
204
1.1 LP TRNH B NH CHIA S DA VO TIN TRNH
To lp tin trnh
C php: id = create_process(N);
Mc ch: to ra N tin trnh v mt tin trnh cha
thc hin cu lnh .
Nh th, c N+1 tin trnh nh nhau c to ra
v mi gi tr ca id c gn tng ng cho
mt tin trnh.
7/17/2010
35
205
V d: to N tin trnh v giao n+1 Nhim v thc hin ssong
id = create_process(N);
switch(id){
case 0: do NhiemVu0 ; break;
. . .
case N: do NhiemVuN ; break;
}
206
b) Hy b tin trnh:
C php: join_process(N, id);
Mc ch: cho php tin trnh c gi tr id trong N tin trnh
khi to c tip tc thc hin nhng phn vic tun t cn
li, cn nhng tin trnh khc kt thc.
Nu ta t sau join_process(N, id) mt cu lnh th cu lnh
ny s khng c thc hin cho n khi tt c cc tin trnh
u thc hin join_process(N, id). Do vn x l song
song khng xut hin m l x l tun t.
207
Vn l cc tin trnh c to lp v truy cp d liu ca
nhau nh th no?
Mt mt mt tin trnh c th mun gi mt phn d liu cc b
cho ring mnh, khng cho nhng tin trnh khc truy cp ti
nhng d liu .
Mt khc, n cng mun trao i thng tin vi cc tin trnh
khc.
X l vn che du hay chia s thng tin nh th no cn
tu thuc vo m hnh m chng ta s dng:
tin trnh (process) hay lung (thread).
Cc tin trnh trong UNIX c s dng nh cc n v tnh
ton c lp. Theo mc nh, vic tnh ton v cp nht b nh
ca mt tin trnh l khng nhn thy c t cc tin trnh
khc.
i vi lung, tt c cc thng tin, theo mc nh, l nhn thy
c.
208
Ch (1/3):
Khi mun s dng b nh chung, Ngi lp trnh cn phi xin
cp pht b nh v sau khi s dng xong phi gii phng
chng.
C hai hm c s thc hin iu ny:
shared(m, &id): xin cp pht m byte b nh chia s cho
tin trnh id.
free_shm(): gii phng b nh c cp.
Cc hm shared(m, &id), free_shm() c gi l hm iu
phi v chia s b nh.
209
V d: Bi ton loi tr nhau.
Xt on chng trnh sau:
main(){
int id, sid, *i, j;
i = (int*)shared(sizeof(int), &sid);
*i = 100; j = 100;
printf(Before fork: %d, %d\n, *i, j);
id = create_process(2);
*i = id; j = id * 2; // (1)
printf(After fork: &d, %d\n, *i, j); // (2)
join_process(3, id);
printf(After join: &d, %d\n, *i, j);
free_shm(sid);
}
To 2 tin trnh v mt tin
trnh cha. Mi gi tr ca id
c gn tng ng cho
mt tin trnh.
Ch tin trnh tng ng vi
gi tr id cn tip tc hot
ng, nhng tin trnh l
kt thc sau li gi hm
trn
Gii phng b nh c
cp
Cu hi: Gi s cu lnh (1) thc hin vi id =2 th cu lnh (2) c in
ra l After fork: 2, 4 hay khng?
Tr li:
Cha chc!!!
Cc gi tr ca *i, j
in ra c th l 2, 4;
4, 4; 6, 4; bi v
khi tin trnh th i
thc hin xong
cu lnh (1), trc
khi n thc hin
(2) th c th cc
tin trnh khc
cp nht li gi tr
ca i.
Lu rng, i l
bin chia s, id l
duy nht (s hiu)
i vi mi tin
trnh.
210
Ch :
Nu c mt tin trnh truy cp vo mt vng nh vi nh cp
nht th n phi c m bo rng khng mt tin trnh no
khc c d liu vng cho n khi vic cp nht kt
thc.
gii quyt c vn trn th phi c c ch m bo
rng, ti mi thi im cc khi lnh ca chng trnh c
thc thi ch bi mt tin trnh.
Nu c mt tin trnh bt u vo thc hin mt khi lnh th
nhng tin trnh khc khng c vo khi lnh .
Khi mt tin trnh vo mt vng lnh no th n s gi kho
(lock).
Ngc li, khi ra khi vng th thc hin c ch m kho
(unlock) cho tin trnh khc c nhu cu s dng.
7/17/2010
36
211
Cc cu lnh thc hin cc yu cu trn:
init_lock(Id): Khi ng b kho vng nh chia s, trong Id
l tn ca vng nh s dng chung.
lock(Id): kho li vng nh Id. Nu mt tin trnh kho mt
vng nh chung th nhng tin trnh khc mun truy cp vo
s phi ch.
unlock(Id): m kho vng b kho v tr li cho tin trnh
khc.
212
S dng c ch gi kho vit li chng trnh trn:
main(){
int *lock1,id, sid1, sid2, *i, j;
lock1 = (int*)shared(sizeof(int), &sid1);
init_lock(lock1);
i = (int*)shared(sizeof(int), &sid2);
*i = 100; j = 100;
printf(Before fork: %d, %d\n, *i, j);
lock(lock1);
*i = id; j = id * 2; // (1)
printf(After fork: &d, %d\n, *i, j); // (2)
unlock(lock1);
join_process(3, id);
printf(After join: &d, %d\n, *i, j);
free_shm(sid1); free_shm(sid2);
}
213
V d (1/2): Chng trnh tnh tng ca hai vector:
for(i = 0; i < N; i++){
C[i] = A[i] + B[i];
}
Thc hin song song ho on chng trnh ny nh th no?
Gi s ta c M tin trnh. Chng ta c th chia N phn t thnh
M phn (gi thit N/M l s nguyn) v gn tng phn cho
mi tin trnh.
Chu trnh trn c th vit thnh:
for (j = id * N/M; j < (id+1)*N/M; j++){
C[j] = A[j] + B[j];
}
P
1
P
2
P
3
P
4
P
5
1 4 7 10 13
2 5 8 11 14
3 6 9 12 15
Trong , id l s hiu ca tin trnh, nhn
gi tr t 0 n M-1. Tin trnh th i x l
N/M phn t lin tip k t i*N/M+1,
Khi N = 15 v M = 5
214
V d (2/2)
Ta c th cho php cc tin trnh truy cp xen k vo cc phn t
ca mng nh sau:
Tin trnh P
i
bt u t phn t th i, sau b qua M phn t
x l phn t tip theo, ngha l n truy cp n i, i+M,
i+2M, v.v
Chu trnh (1) khi c vit nh sau:
for(j = id; j < N; j+=M){
C[j] = A[j] + B[j];
}
P
1
P
2
P
3
P
4
P
5
1 2 3 4 5
6 7 8 9 10
11 12 13 14 15
Khi N = 15 v M = 5
215
Mt tin trnh l bc tranh v s hot ng ca mt ch.trnh.
Mi tin trnh bao gm mt hoc nhiu lung.
Cc lung c th xem nh cc tp con ca mt tin trnh.
Cc lung ca mt tin trnh c th chia s vi nhau v
khng gian a ch, cc on d liu v mi trng x l,
ng thi cng c vng d liu ring thao tc.
Cc tin trnh v cc lung trong h thng song song cn
phi c ng b, nhng vic ng b cc lung c thc
hin hiu qu hn i vi cc tin trnh.
ng b cc tin trnh i hi tn thi gian hot ng ca h
thng, trong khi i vi cc lung th vic ng b ch yu
tp trung vo s truy cp cc bin chung ca chng trnh.
? tin trnh lung
1.2 LP TRNH B NH CHIA S DA VO LUNG
216
. Nhiu h iu hnh hin nay h tr a lung nh:
SUN Solaris
Window NT
OS/2, v.v.
Bn trong nhng h iu hnh ny, nhng c tnh h tr
cho NSD khai thc c cc lung trong chng trnh ca
mnh thng khc nhau.
Hin nay c mt chun, l Pthread ca IEEE Portable
Operating System Interface, POSIX.
7/17/2010
37
217
Thc hin cc lung ca Pthread
Trong Pthread, chng trnh chnh cng chnh l mt lung.
Mt lung c th c khai bo, to lp v kt thc bng nhng
chng trnh con sau:
pthread_t aThread; // Khai bo mt lung
pthread_create(&aThread,&status,(void*)proc1,(void*)arg);
pthread_join(aThread, void *status);
218
1.2 LP TRNH CHIA S B NH DA VO LUNG
pthread_create(&aThread,&status,(void*)proc1,(void*)arg);
pthread_join(aThread, void *status);
aThread
proc1(&arg);
{ .
.
.
Return(*status)
}
Hot ng ca mt lung trong Pthread
219
V d: Xt mt chng trnh n gin s dng cc lung.
Chng trnh gm hai chng trnh con reader() v processor().
Reader() c dy d liu t lung vo v processor() x l trn
cc d liu c vo.
xem gio trnh
220
. Vn thc hin loi tr nhau gia cc lung
Vn thc hin loi tr nhau gia cc lung cng tng t
nh i vi cc tin trnh.
C bn hm nguyn thu:
pthread_mutex_init(mutex, NULL)
pthread_mutex_lock(mutex)
pthread_mutex_unlockinit(mutex)
pthread_mutex_destroy(mutex)
V d: Chng trnh c vo mt mng a[ ] cc s nguyn v
thc hin cng song song (theo nhiu lung) cc phn t ca
mng vi mt hng s k (xem gio trnh).
221
III. M HNH LP TRNH
Assignment: Parallel Programming with PCN
Seyd H.Roosta, Parallel Processing and Parallel Algorithms,
Pages 149-164
http://books.google.com/books?id=pi6
dl5eWJpkC
222
Seyd H.Roosta, Parallel Processing and Parallel Algorithms,
Pages 140-149
http://books.google.com/books?id=pi6dl5eWJpkC
Assignment: Parallel Programming with UNIX
7/17/2010
38
223
Assignment: POSIX Threads Programming
https://computing.llnl.gov/tutorials/pthreads/
http://www.yolinux.com/TUTORIALS/LinuxTuto
rialPosixThreads.html
http://www.humanfactor.com/pthreads/
224
Assignment: Parallel Programming with Thread in JAVA
http://java.sun.com/docs/books/tutorial/essential/threads/
http://courses.coreservlets.com
225
Thread in JAVA
Black coffee for today -
- Thread
http://java.sun.com/docs/books/tutorial/essential/threads/ 226
Thread in JAVA
L c ch iu khin bng NNLT th hin s song
song trong mt chng trnh.
Cho php ci t cc chng trnh thc hin nhiu tc
v cng mt lc (v d?)
Lung cn c th c
To lp (tng thm n v song song)
ng b ha (iu phi)
Hy b (gim bt n v song song)
Trong Java, lung l mt loi i tng h thng:
Cc phng thc trn i tng lung
Mi i tng l mt n v song song c th c
thc hin mt cch c lp
227
Li ch?
ng dng my khch/n ngi dng:
Ti d liu / trang Web t mng
p ng s kin GUI trong lc x l IO / Database
Tng tc x l khi c nhiu b x l
ng dng my ch:
a kt ni
Kh g ri v bo tr, gii hn tnh kh chuyn
(portable)
228
X l lung trong Java
Java l ngn ng lp trnh hng i tng h tr a lung,
tin li cho cc ng dng web.
Trong m hnh hng i tng, tin trnh v th tc l th
yu, mi chc nng ca chng trnh c xc nh thng
qua cc i tng.
Cng ging nh tin trnh, lung c to lp, sau thc
hin mt s cng vic v kt thc hot ng khi khng cn vai
tr s dng.
7/17/2010
39
229
Hai hng tip cn Threads trong JAVA
1. Xy dng lp con ca lp Thread.
(Trong Java c mt lp c xy dng sn l Thread,
S dng Thread lm lp c s xy dng nhng lp k tha
(lp con) mi.
2. Ci t giao din Runnable.
See also:
http://java.sun.com/docs/books/tutorial/essential/threads/
230
b. To lp cc lung
class MyClass extends Thread{
// Mt s thuc tnh
public void run(){
// Cc lnh cn thc hin theo lung
}
// Mt s phng thc khc c vit hay b sung
}
Khi chng trnh chy n s gi mt phng thc c bit
c khai bo trong Thread l start() bt u mt lung
c to ra.
231
b. To lp cc lung
Java khng h tr a k tha. Do , nu ngi lp trnh
mun to ra mt lp k tha t mt lp c s v thc hin
c theo lung th n cng ng thi phi k tha t lp
Thread. iu ny khng thc hin c.
Java gii quyt hn ch trn bng cch xy dng khi nim
Interface. Giao din h tr thc hin theo lung l Runnable.
Ngi lp trnh thit k cc lp thc hin theo lung bng
cch ci t theo giao din Runnable.
class MyClass implemnets Runnable{
. . .
}
232
c. Cc trng thi ca Thread
Mt lung c th mt trong cc trng thi sau:
new: khi mt lung mi c to ra vi ton t new().
ready: khi chng ta gi phng thc start() bt u ca mt
lung.
blocked: t trng thi runnable chuyn sang trng thi b chn
khi gi mt trong cc phng thc: sleep(), suspend(), wait(),
hay b chn li Input/output.
dead: lung chuyn sang trng thi cht khi n kt thc hot
ng bnh thng, hoc gp phi ngoi l khng thc hin
tip c.
Hnh di y m t s chuyn trng ca cc lung trong h
thng.
233
234
Phng thc yield() s ngng lung hin hnh cho
mt lung khc c cng u tin chy
wait(): ging yield(), nhng yu cu mt lung khc phi
nh thc n
while (!condition) wait();
notify(): Lung no c th nh hng n condition s
gi notify() phc hi lung ang ch
Lp trnh vin phi chu trch nhim bo m mi wait() c
mt notify() tng ng
7/17/2010
40
235
d. iu g xy ra khi c Lung mi?
Lung chnh vn tip tc
Lung mi thi hnh phng thc run() v kt thc khi
phng thc kt thc.
Nu c bt k lung no gi System.exit(0) th n s
gii phng tt c mi lung.
C th xem run() nh l phng thc chnh ca ring
mi lung
236
e. Chm dt mt Lung
Lung kt thc khi phng thc run() kt thc
Tuy nhin, iu g xy ra khi lung ang ngh
(sleeping) hoc ang b kha (blocked)?
y l lc m vai tr ca phng thc interrupt() c
th hin. Khi interrupt() c gi trn mt Lung ang
b kha, lung s b chm dt.
237
f. Cc vn khc v Lung
Chia s v ng b ha
Cc lung c th cng truy cp n cc i tng
c kt hp vi mt tin trnh n.
Lp lch
Nu (# lung) != (# vi x l), th vic lp lch cho cc
lung l mt vn
Cc thao tc ca cc lung khc nhau c th xy ra
theo th t bt k
Cc lung c th c thit lp u tin
238
V d: s dng Thread trong JAVA tnh tng cc phn
t ca mt mng.
Xem gio trnh
239
2. TNH TON PHN TN
nh ngha: Tnh ton phn tn l nhng tnh ton c thc
hin trn c s kt hp tnh ton v truyn thng ca hai hay
nhiu my tnh trn mng.
u im:
Cho php chia s d liu c lu nhiu my tnh khc
nhau (khng c b nh chia s).
Chia s vi nhau v mt s chc nng chnh ca my tnh.
tin cy v kh nng th li cao hn. Trong trng hp c
mt my tnh b s c th nhng my tnh khc c th thay th
hon thnh nhim v ca h thng.
Nhc im:
Nhng vn lin quan n vic qun tr h thng, nh v ti
nguyn, vn m bo an ton, an ninh thng tin, v.v. khng
bng h tp trung
240
2. TNH TON PHN TN: M HNH TRUYN THNG IP
4.2.1 M hnh truyn thng ip (Message Passing)
Cc n v XLSS trong m hnh truyn thng ip l cc
tin trnh.
Cc tin trnh c th thc hin trn nhng b x l khc
nhau v khng truy cp c vo khng gian a ch chia
s.
Ch c knh truyn l c th chia s cho cc tin trnh,
thng l LAN hoc mng din rng.
Hin nay c nhiu cng c lp trnh c s dng cho tnh
ton phn tn nhiu mc tru tng khc nhau, nh
PVM, MPI, DCE, v.v.
Trong cc h thng phn tn khng c b nh chia s
trao i d liu vi nhau vic trao i c thc hin bng
cch truyn thng ip. M hnh Client-Server cng c th
s dng c ch ny ci t.
7/17/2010
41
241
II. TNH TON PHN TN: M HNH TRUYN THNG IP
1. M hnh truyn thng ip (Message Passing)
Vic truyn thng v ng b ho hot ng ca cc tin trnh
c thc hin thng qua hai phng thc send() v
receive().
Tt c cc bin l bin cc b ca cc tin trnh. V th, nhng
vn v xung t d liu (cn phi kho d liu khi mt tin
trnh truy cp), hay tranh chp thng tin khng xut hin trong
m hnh ny.
Vic ng b ho cc tin trnh ca mt chng trnh song
song cng c thc hin theo c ch truyn thng ip.
Ngha l, khi mt tin trnh mun gi mt thng ip th n
phi ch cho n khi tin trnh nhn sn sng nhn thng ip
v ngc li, cng tng t. y khng c cc buffer
tm lu gi cc thng ip cn trao i gia cc tin trnh.
242
2. M hnh tng qut
Trong m hnh truyn thng ip, cc tin trnh chia s vi
nhau knh truyn thng.
Cc knh c truy cp bi hai phng thc: send() v
receive().
bt u trao i c vi nhau th mt tin trnh phi gi
i (send) mt thng ip vo knh truyn v c mt tin trnh
khc yu cu nhn (receive) thng ip .
S trao i c hon tt khi d liu c chuyn t ni
gi ti ni nhn.
243
2. M hnh tng qut
C hai loi m hnh truyn thng ip: d b v ng b
a.Truyn thng ip d b:
Trong m hnh ny, mt knh truyn c gi thit l c kh
nng tip nhn khng b gii hn (bng cch s dng buffer)
tip nhn cc thng ip gi n cho mi tin trnh.
Do , tin trnh gi s khng phi ch tin trnh nhn sn
sng nhn m c gi khi c yu cu.
Hai tin trnh gi v nhn c th hot ng gn nh c lp
nhau v thng ip c th nhn c sau mt khong thi
gian no (lu bt k) k t khi n c gi i.
Tin trnh nhn th phi ch cho n khi c c thng ip
ca mt tin trnh khc gi cho n.
244
Mt s vn ny sinh trong d b
Khi tin trnh A gi mt thng ip cho tin trnh B th sau
A cn phi bit xem B c nhn c hay khng. Ngha l A
phi ch nhn c cu tr li khng nh ca B.
Vic phn pht thng ip cng khng th m bo rng
khng b tht bi. Nu A gi i mt thng ip cho B v
khng nhn c cu tr li th n cng khng c cch no
bit c l thng ip c gi n ch cha hoc
tin trnh B b hu b trong qu trnh x l hoc ngay c khi
cu tr li ca B khng n c A.
Cc thng ip u phi a vo b m (hng i), nhng
trong thc t khng gian hng i l hu hn. Khi c qu
nhiu thng ip c gi i th hoc chng trnh s
khng thc hin c hoc phng thc gi b chn li.
iu ny vi phm ng ngha ca m hnh truyn thng ip
d b.
245
b.Truyn thng ip ng b:
Trong m hnh ny, tin trnh gi b chn li cho n khi
tin trnh nhn sn sng nhn. y, s truyn thng v
ng b ho lun gn cht vi nhau. M hnh ny c th
tm thy nhng h thng a b x l c ci t da
trn cc Transputer.
H thng truyn thng ip ng b hon ton ging nh
h in thoi, knh truyn b chn li trong qu trnh m
thoi. H truyn d b li ging nhiu hn vi h thng bu
chnh, ngi nhn phi ch cho n khi c th c gi
n.
Hu ht cc th vin lp trnh h tr m hnh truyn thng
ip d b.
246
III. M HNH LP TRNH
(Trong phn ny chng ta nh ngha m hnh lp trnh cho h thng
truyn thng ip d b.)
Lp trnh theo m hnh truyn thng ip trong h thng nhiu
my tnh c th thc hin theo ba cch:
1. S dng NNLT song song c bit, v d Occam c thit k
s dng vi cc Transputer (Inmos 1986)
2. S dng NNLT bc cao (tun t) truyn thng c m rng
bng cch b sung thm cc t kho v m rng c php
x l vic trao i thng ip, v d CC++ (m rng ca C++)
3. S dng nhng NNLT bc cao v cung cp h thng chng
trnh th vin gm nhng th tc x l vic trao i thng
ip, v d ngn ng C v h chng trnh th vin chy
vi PVM.
Chng ta tp trung nghin cu cch th ba.
7/17/2010
42
247
III. M HNH LP TRNH
1. Pht sinh cc tin trnh con (1/5)
Mc ch:
thc hin nhng cng vic con ca mt chng trnh song
song.
Vic pht sinh tin trnh con thng xy ra khi mt chng
trnh bt u thc hin nh mt tin trnh v sau pht sinh
ra nhiu tin trnh con khai thc kh nng song song ca
bi ton.
C hai cch to lp tin trnh: to lp tnh v to lp ng.
248
III. M HNH LP TRNH
a.To lp tin trnh tnh:
Tt c cc tin trnh phi c xc nh trc khi thc hin
Trong h thng c mt s tin trnh c xc nh trc:
mt tin trnh iu khin cn c gi l tin trnh ch
(master),
nhng tin trnh khc c gi l tin trnh t (slave).
Sau khi chng trnh ngun c vit vi cc lnh phn chia
cng vic cho tng tin trnh, n s c dch sang m thc
thi c cho nhng tin trnh .
Qu trnh ny c m t nh hnh sau:
249
III. M HNH LP TRNH
Chng
trnh
ngun
on
chng
trnh
thc thi
on
chng
trnh
thc thi
BXL 0
BXL n-1
Bin dch theo
cc b x l
Dch n chng trnh, a thao tc d liu
H th vin MPI c xy dng theo cch to lp tnh nh trn
250
III. M HNH LP TRNH
b.To lp tin trnh ng:
Cc tin trnh c to lp sau chng c th thc hin
trong cc tin trnh khc.
Cc tin trnh c th c to lp mi, b hu b c iu kin
hoc mt s tin trnh c th thay i trong qu trnh thc
hin.
Kin trc cho php thc hin to lp tin trnh ng l MPMD
(MIMD), trong nhng chng trnh khc nhau c th thc
hin trn nhng b x l khc nhau.
251
III. M HNH LP TRNH
Tt c cc h th vin truyn thng ip u c hm pht sinh
cc tin trnh con vi cu trc tng qut nh sau:
spawn(prog: string, host: int, count: int, id[]:int)
Trong ,
prog: tn chng trnh c np xung nh l chng trnh
con.
Kt qu khi thc hin thnh cng s tr li mng id l a ch
ca cc tin trnh c to ra truyn thng ip. Mng id
c ch s t 1, tin trnh u tin ca chng trnh song song
c ch s l 0.
host: tn ca tin trnh m cc tin trnh con c to ra.
count: s cc i din ca chng trnh c to ra.
252
III. M HNH LP TRNH
Tiu lun: tm hiu v MPI:
http://www-unix.mcs.anl.gov/mpi/
http://www.mpi-forum.org/
http://en.wikipedia.org/wiki/Message_Passing_Interface
7/17/2010
43
253
ASSIGNMENT: Parallel Programming with MPI
http://www.cs.usfca.edu/mpi/ http://www.mcs.anl.gov/learning.html
http://www-unix.mcs.anl.gov/mpi/ http://www.mpi-forum.org/
254
III. M HNH LP TRNH
2. Ng ngha ca knh truyn (1/2)
Nhng h thng khc nhau s cung cp nhng knh truyn
nhng mc tru tng khc nhau.
Hu ht cc h thng u m bo rng cc knh truyn
thng khng c cc li thng thng. Ngha l, tt c cc
thng ip c gi i v s n c ch m khng b sai
lc.
Knh truyn thng c th c xem nh l hng i, v thng
ip s c gi n ch theo th t n c gi i.
Knh truyn thng cng c th c xem nh l hp th
(mailbox) ca mt tin trnh nhn/gi thng ip. mc tru
tng ny s loi b c rng buc v th t n ca cc
thng ip.
Tin trnh c th chn cc thng ip t hp th theo mt s
tiu ch no .
255
III. M HNH LP TRNH
2. Ng ngha ca knh truyn (1/2)
Mt s h thng a ra khi nim thng ip c cu trc. Cc
knh c nh ngha theo cc kiu ca thng ip v mt s
knh c bit ch truyn ti c mt s kiu nht nh.
Mt tin trnh mun gia nhp vo mt knh c kiu xc nh
th phi t gn b vi knh .
Mt knh cng c th phc v cho mt nhm cc tin trnh
trong mt chng trnh song song.
Cc knh c m t nh sau:
1. Khai bo mt knh c gn vi kiu
channel mych(id1:type1, id2:type2, , idn:typen)
Knh mych chuyn ti c nhng thng ip c xc nh
nh trong cc i s.
2. Gn tin trnh vi knh truyn
join_channel(mych: string);
Gi tin trnh hin thi gia nhp vo knh mych.
256
III. M HNH LP TRNH
3. Cc hm send() v receive()
Vic gi mt thng ip c thc hin bng cch xc nh
a ch ca mt hay tt c cc tin trnh nhn theo mt knh
truyn.
Vic la chn thng ip nhn c th da vo tin trnh
gi, knh truyn, hay th bi (tag) ca thng ip, v.v.
Li gi send() l khng b chn, khng phi ch cu tr li
ca ni nhn. Nhng li gi receive() s b chn li, ch c
tip tc khi c mt thng ip tng ng c gi i.
257
III. M HNH LP TRNH
3. Cc hm send() v receive()
Gi thng ip cho mt tin trnh id:
send(id: int, message: message_type);
send(id: int, tag: int, message: message_type);
Gi thng ip ti mt knh: mt thng ip c th gi cho tt
c cc tin trnh trn cng mt knh mych.
send(mych: channel, message: message_type);
Nhn thng ip t mt knh: nhn mt thng ip ang
ch i t mt knh th c th s dng li gi hm sau:
receive(mych: channel, message: message_type);
Nu thng ip c ghi th th tin trnh nhn c th phn
loi thng ip trong hp nhn v chn thng ip theo th
xc nh.
receive(id: int, tag: int, msg: message_type);
258
III. M HNH LP TRNH
V d: trong C chng ta c th gi
send(&x, destination_id); { tin trnh ngun}
receive(&y, source_id); { tin trnh ch}
gi gi tr d liu x t tin trnh ngun (source-id) sang
bin y cho tin trnh ch.
x
.
.
.
send(&x, 2);
.
.
.
y
.
.
.
receive(&y, 1);
.
.
.
Tin trnh 1
Tin trnh 2
S trao i thng ip gia hai tin trnh
7/17/2010
44
259
III. M HNH LP TRNH
4. Truy vn trn knh
Mc ch: kim tra thng ip gi i c n ch hay khng?
gii quyt vn ny, hu ht cc chng trnh th vin
cung cp cc hm truy vn bit cc trng thi ca knh
truyn.
1. Kim tra xem trn knh c nhng thng ip gi n cho tin
trnh?
empty(ch: channel);
2. Hm gi xc nh xem thng ip ang ch nhn c
phi c gi t tin trnh id v c th tag?
probe(id: int, tag: int);
260
III. M HNH LP TRNH
5. Truyn thng theo nhm
Mc ch: Nhiu chng trnh cn pht tn v nhn d liu t
nhiu tin trnh phn tn, ngha l cn trao i vi tng nhm
trong chng trnh song song.
Cc hm thc hin truyn thng theo nhm:
Broadcast(mych:channel,tag:int, msg:message_type); pht tn
cng mt thng ip cho tt c cc tin trnh trn knh mych.
D liu
buf
. .
.
Broadcast();
.
.
.
D liu
. .
.
Broadcast();
.
.
.
D liu
. .
.
Broadcast();
.
.
.
Tin trnh 0 Tin trnh 1 Tin trnh n-1
Tt c cc tin trnh u gi hm Broadcast(). Hnh ng pht tn d liu s
khng thc hin c cho n khi tt c cc tin trnh u thc hin li gi
Broadcast().
Cc tin trnh tham gia
trao i trong pht tn d
liu phi c xc nh.
tin trnh s 0 c xem
nh tin trnh gc cha
d liu mng buf
pht tn cho nhng tin
trnh khc.
261
III. M HNH LP TRNH
Reduce(): thc hin php ton s hc/logic trong nhm cc tin
trnh v gi kt qu ti tin trnh gc.
Reduce(mych:channel,op:op_type, res:Result_type,
root: int,tag:int, msg:message_type);
Trong ,
Op_type = {MAX, MIN, SUM, PROD, LAND, LOR, BAND,
BOR, LXOR, BXOR}}
262
III. M HNH LP TRNH
V d: s di y m t hm Reduce() tp hp cc gi tr t
n tin trnh v thc hin php cng (SUM) tin trnh gc.
D liu
buf
. . .
Reduce();
.
.
.
D liu
. . .
Reduce();
.
.
.
D liu
. . .
Reduce();
.
.
.
+
263
III. M HNH LP TRNH
Scatter(): phn tn cng vic cho cc tin trnh. D liu mng
buff c chia nh thnh n on v phn tn cho n tin trnh
trn knh mych.
Scatter(mych:channel, n:int, Buff[N]:DataType);
Hm ny c s dng gi phn t th i ca mt mng d
liu ti cho tin trnh th i.
D liu
buf
. .
.
Scatter();
.
.
.
D liu
. .
.
Scatter();
.
.
.
D liu
. .
.
Scatter();
.
.
.
264
III. M HNH LP TRNH
Gather(): d liu c gi i theo hm Scatter() c x l bi
nhng tin trnh nhn c v sau c tp hp li cho
mt tin trnh.
Gather(mych:channel,Buff[N]:DataType,root:int,sendbuff[N]:Data
Type);
Ngc li hm Scatter(), d liu t tin trnh th i c nhn v
tin trnh gc v c a vo phn t th i ca mng buf,
D liu
buf
. . .
Gather();
.
.
.
D liu
. . .
Gather();
.
.
.
D liu
. . .
Gather();
.
.
.
7/17/2010
45
265
III. M HNH LP TRNH
Barrier(): thc hin vic ng b ho nhng tin trnh cng gia
nhp mt knh truyn. Mi tin trnh phi ch cho n khi tt
c cc tin trnh khc trn knh t n im ng b ho
bng li gi Barrier() trong chng trnh.
Barrier(mych:channel);
266
IV. LP TRNH TRN CM MY
267
IV. LP TRNH TRN CM MY TNH
Nhn xt 1:
M hnh truyn thng ip c s dng rt hiu qu lp
trnh song song theo cm my tnh.
PVM cung cp mi trng phn mm truyn thng ip
cho cc cm my tnh thun nht v c khng thun nht.
PVM c mt tp hp cc hm th vin c vit bng C
hoc Fortran.
Mi chng trnh c vit bng C, c bin dch chy
trn nhng kiu my tnh xc nh trn mng.
Mi nt trong mng (LAN) phi truy cp n h thng tp
cha cc chng trnh c dch thc hin.
268
Nhn xt 2:
Cm my tnh trong PVM phi c xc nh theo cc mc
u tin thc hin cc chng trnh.
Cch thc hin tt nht l to ra mt danh sch tn gi ca
cc my tnh v t hostfile. Tp ny c PVM c
thc hin cc chng trnh.
Mi my tnh c th chy mt hay nhiu tin trnh (chng
trnh ng dng).
Cc chng trnh ng dng chy trong cc my tnh thng
qua cc tin trnh ca PVM trao i vi nhau trn mng.
Cc tin trnh PVM yu cu thng tin chn la ng
truyn d liu.
269
PVM
Ch.trnh
ng dng
PVM
Ch.trnh
ng dng
PVM
Ch.trnh
ng dng
My tnh
trm
My tnh
trm
My tnh
trm
Trao i thng
ip trn mng
S trao i thng ip ca cc my tnh trong h PVM
270
Cc chng trnh ca PVM thng c t chc theo m
hnh ch-t (master-slave), trong tin trnh ch c thc
hin trc tin, sau cc tin trnh t s c to ra trong
tin trnh ch .
Mt s hm thng dng trong PVM:
pvm_spawn(): pht sinh tin trnh mi trong PVM
pvm_mytid(): ghi danh tin trnh mun tham gia vo h PVM
pvm_exit(): hu b tin trnh
pvm_send() v pvm_recv(): Cc chng trnh trao i thng
ip vi nhau.
7/17/2010
46
271
Tt c cc th tc gi u khng b chn (d b) cn cc th
tc nhn th hoc b chn (c ng b) hoc khng b chn.
Cc thao tc chnh ca vic gi v nhn d liu c thc
hin cc b m buffer.
Nu d liu c gi i l mt danh sch cc mc c cng
kiu th trong PVM s dng
pvm_psend() v pvm_precv().
272
Hnh di y m t hot ng ca hai tin trnh trao i mt
mng d liu vi nhau.
.
.
.
pvm_precv();
.
.
.
Tin trnh 2 send buffer
.
.
.
pvm_send();
.
.
.
Tin trnh 1
Ch thng
ip
Tip tc
x l
Gi hm pvm_psend() v pvm_precv()
273
Khi d liu chuyn i l phc tp, gm nhiu kiu khc nhau
th chng phi c ng gi li (pack) gi n buffer, sau
tin trnh nhn li m gi (unpack) nhn v d liu
tng ng. l cc hm:
pvm_pkint() v pvm_upkint() dng cho d liu kiu int
pvm_pkfloat() v pvm_upkfloat() dng cho d liu kiu float
pvm_pkstr() v pvm_upkstr() dng cho d liu kiu string, v.v.
Lu : th t m gi ly d liu ra phi ng theo th t m
chng c ng gi tin trnh gi. B m buffer gi
d liu l mc nh v n phi c khi to tin trnh gi
bng lnh pvm_intend().
274
Tng t, cc lnh khc v trao i thng ip theo nhm nh:
pvm_bcast()
pvm_scatter()
pvm_gather()
pvm_reduce(), v.v.
Xem v d ti liu
275
Tiu lun: tm hiu v PVM
http://www.csm.ornl.gov/pvm/manpages.html
http://www.netlib.org/pvm3/
276
V. NH GI CHNG TRNH SONG SONG
Thi gian thc hin song song t
p
= t
comp
+ t
comm
Trong ,
t
comp
: thi gian tnh ton ; t
comm
: thi gian truyn thng.
Thi gian tnh ton t
comp
c xc nh ging nh thut ton
tun t.
Khi c nhiu tin trnh thc hin ng thi th ch cn tnh thi
gian thc hin ca tin trnh phc tp nht.
Trong phn tch phc tp tnh ton, chng ta lun gi thit
rng, tt c cc b x l l ging nhau v thao tc cng mt
tc nh nhau. i vi nhng cm my tnh khng thun
nht th iu ny khng m bo. Do , vic nh gi thi
gian tnh ton ca nhng h nh th l kh.
7/17/2010
47
277
V d:
Gi s cn cng n s trn hai my tnh, trong mi my
cng n/2 s vi nhau v lu my tnh ca mnh.
Kt qu ca my tnh th hai khi c tnh xong s c
chuyn v my tnh th nht n cng hai kt qu b
phn vi nhau.
278
Bi ton c pht biu nh sau:
1. My tnh 1 gi n/2 s cho my tnh 2
2. C hai my tnh cng n/2 s mt cch ng thi
3. My tnh 2 chuyn kt qu tnh c v my tnh 1
4. My tnh 1 cng hai kt qu c kt qu cui cng.
Thi gian tnh ton ( bc 2 v 4): t
comp
= n/2 + 1
Thi gian truyn thng ( bc 1 v 3):
t
comm
= (t
startup
+ n/2 * t
data
) + (t
startup
+ t
data
)
= 2*t
startup
+ (n/2 + 1) * t
data
phc tp tnh ton l O(n) v phc tp truyn thng
cng l O(n), do vy phc tp chung ca thut ton l
O(n).
279
Bi tp
4.1 Cho on chng trnh
printf(I am here\n);
printf(%d is here\n, id);
Bao nhiu dng in ra is here, s hiu ca tin trnh in ra
c theo mt trt t c nh qua cc ln thc hin hay
khng?
4.2 Vit mt on chng trnh to ra hai tin trnh v
thc hin tnh 2+4+6+8 song song.
280
Bi tp
4.3 Vit chng trnh song song vi phc tp O(log n)
tnh gi tr ca a thc
F = a
0
x
0
+ a
1
x
1
+ a
2
x
2
+ + a
n-1
x
n-1
4.4 Vit chng trnh song song tnh n! s dng hai
tin trnh thc hin ng thi. Mi tin trnh tnh gn mt
na dy kt qu v mt tin trnh ch kt hp hai kt
qu li.
4.5 Vit chng trnh song song tm ra tt c cc s
nguyn t nh hn hoc bng N.
4.6 Vit chng trnh song song theo lung (Thread)
tnh xc nh gi tr cc i ca N phn t.
281
Bi tp
4.7 Vit chng trnh song song theo lung trong m hnh
chia s b nh thc hin nhn hai ma trn cp nn.
4.8 Xy dng h x l hnh ng tnh sin x
sin x = x x/3 + x/5 x/7 + x/9 + . . .
4.9 Cho bit kt qu in ra ca on chng trnh sau:
j = 0;
k = 0;
forall (i = 1; i <= 2; i++){
j += 10;
k += 100;}
printf(i = %d, j = %d, k = %d\n,i,j,k)
282
Bi tp
4.10 Ngi ta vit on chng trnh sau thc hin
chuyn v ma trn
forall (i = 1; i < n; i++)
forall(j = 0; j < n; j++)
a[i][j] = a[j][i];
on chng trnh trn c thc hin c khng? nu khng
thc hin c th hy vit li chng trnh khc thc
hin c vic chuyn v ma trn.
7/17/2010
48
283
CHNG 5. THUT TON SONG SONG
NI DUNG
1. Nguyn l thit k thut ton song song
2. Cc cch tip cn trong thit k TTSS
3. Phn tch v nh gi thut ton song song
4. Cc thut ton sp xp, tm kim song song
5. Mt s thut ton song song thng dng.
284
5.1 NGUYN L THIT K THUT TON SONG SONG
Nhng thut ton, trong c mt s thao tc c th thc
hin ng thi c gi l thut ton song song.
Thut ton song song?
Tng qut, thut ton song song l mt tp cc tin trnh
(process) hoc cc tc v (task) c th thc hin ng thi
v c th trao i d liu vi nhau kt hp cng gii
mt bi ton t ra.
285
05 cu hi t ra trc khi thit k mt thut ton song song?
1. Vic phn chia d liu cho cc tc v nh th no?
2. D liu c truy cp nh th no,
3. Nhng d liu no cn phi chia s?
4. Phn cc tc v cho cc tin trnh (b x l) nh th no?
5. Cc tin trnh c ng b ha ra sao?
286
Cc nguyn l c bn trong thit k TTSS (1/2)
1. Nguyn l lp lch:
Gim ti thiu cc b x l s dng trong thut ton sao
cho thi gian tnh ton l khng tng (xt theo kha cnh
phc tp).
Ngha l, nu phc tp tnh ton ca thut ton l
O(f(n)) th thi gian thc hin ca chng trnh c th
tng khi s b x l gim, v thi gian tnh ton tng th
tng ln mt hng s no - nhng vn l O(f(n)).
2. Nguyn l hnh ng: Nguyn l ny c p dng khi
bi ton xut hin mt dy cc thao tc {T
1
, T
2
, . . ., T
n
},
trong T
i+1
thc hin sau khi T
i
kt thc.
287
Cc nguyn l c bn trong thit k TTSS (2/2)
3. Nguyn l chia tr:
Chia bi ton thnh nhng phn nh hn tng i c
lp vi nhau v gii quyt chng mt cch song song.
4. Nguyn l th ph thuc d liu:
Phn tch mi quan h d liu trong tnh ton xy dng
th ph thuc d liu v da vo xy dng thut
ton song song.
5. Nguyn l iu kin tng tranh:
Nu hai tin trnh cng mun truy cp vo cng mt mc
d liu chia s th chng phi tng tranh vi nhau, ngha
l chng c th cn tr ln nhau.
288
Ngoi ra khi thit k TTSS cn phi quan tm
Cu hnh t p lin kt mng: cng mt thut ton song
song ci t trn hai my tnh c cu hnh t p lin kt
mng khc nhau th c th c phc tp khc nhau.
V d: DAP l my tnh kiu SIMD vi 64 * 64 b x l, thi
gian nhn ma trn l tuyn tnh theo kch c ca ma trn
v ph thuc vo ng truyn d liu gia cc hng vi
ct.
Nhiu thut ton song song c thit k da trn nhng
kin thc v kin trc my tnh, ngn ng lp trnh song
song v cc phng php tnh ton.
7/17/2010
49
289
5.2 CC CCH TIP CN TRONG THIT K TTSS
C ba cch tip cn thit k thut ton song song:
1. Thc hin song song ho nhng thut ton tun t, bin
i nhng cu trc tun t tn dng c nhng kh
nng song song t nhin ca tt c cc thnh phn trong
h thng x l.
2. Thit k nhng thut ton song song mi ph hp vi kin
trc song song.
3. Xy dng nhng thut ton song song t nhng thut ton
song song c xy dng cho ph hp vi cu hnh
tp v mi trng song song thc t.
Nhn xt: Cch 1 v 2 l thng dng nht
290
5.2 CC CCH TIP CN TRONG THIT K TTSS
Ch :
khi chuyn mt thut ton tun t sang thut
ton song song hoc chuyn mt TTSS thch
hp vi kin trc ang c.
Cn tr li hai cu hi:
1. Kin trc no ph hp cho bi ton?
2. Nhng bi ton loi no s x l hiu qu
trong kin trc song song cho trc?
291
5.3 PHN TCH V NH GI THUT TON SONG SONG
phc tp:
- Thi gian
- Khng gian
phc tp:
- Tun t
- Song song
292
Tun t?
nh ngha: Mt thut ton c phc tp tnh ton
f(x) = O(g(x)) - C>0, x
0
eN sao cho 0 f(x) C*g(x), vi
mi s lng d liu vo x x
0
.
V d: cho f(n) = 1+2+...+n
V f(n) < n + n + ... + n = n
2
nn f(n) l O(n
2
) vi C=x
0
=1
Song song?
phc tp tnh ton ca TTSS khng ch ph thuc vo kch
c ca d liu u vo m cn ph thuc vo kin trc my
tnh song song v s lng cc b x l c php s dng
trong h thng.
293
Song song?
phc tp theo thi gian ca TTSS s dng p b x l
gii mt bi ton c kch c n l hm f(n,p) xc nh thi
gian cc i gia thi im bt u thc hin thut ton bi
mt b x l v thi im kt thc ca cc b x l i vi
b d liu vo bt k.
C hai loi php ton khc nhau trong cc TTSS:
Cc php ton c s nh +, -, *, /, AND, OR, v.v.
Cc php ton truyn d liu trn cc knh truyn.
294
Cc nh ngha
nh ngha 1 (nh l Brent):
Mt thut ton song song c phc tp tnh ton O(T) vi
P b x l khi n thc hin nhiu nht l O(T* P) php ton
c s.
(gii hn s lng php ton c s c thc hin ca
mt thut ton c phc tp cho trc)
7/17/2010
50
295
Cc nh ngha
nh ngha 2:
Mt thut ton song song c phc tp tnh ton O(T) s
dng nhiu b x l thc hin O(e) php ton c s th
khi ci t vi P b x l s c phc tp thi gian l
O([e/P]+ T).
nh ngha 2 ch ra rng khi s b x l c s dng gim
xung trong mt phm vi nht nh th thut ton tip tc
lm vic nhng thi gian thc hin s tng ln.
296
Cc nh ngha
nh ngha 3:
Mt thut ton song song c phc tp tnh ton O(T) vi
P b x l c th ci t vi [P/p], 1 p P b x l th s
c phc tp thi gian l O(p*T).
nh ngha 3 khng nh rng c cch ci t thut ton
song song khi s cc b x l c s dng b gim
xung.
297
Mc song song ca thut ton:
l s lng cc i cc php ton c lp c th thc hin
ng thi mi thi im thc hin ca thut ton.
H s gia tc: bit mc song song ha ca TTSS
H s gia tc ca TTSS s dng p b x l c xc nh:
Trong ,
+ T
S
l thi gian thc hin tnh ton trn mt b x l
+ T
p
l thi gian thc hin tnh ton trn p b x l.
Vi gi thit l b x l tun t v cc b x l song song l
nh nhau.
p
s
p
T
T
S =
298
P: lp cc bi ton c th gii c trong thi gian a thc
NP l lp cc bi ton, m mi nghim gi nh u c th
c kim chng trong thi gian a thc.
Hin nhin P_ NP.
Gi C l bi ton kh nht trong NP. C c gi l NP-y
.
5.4 NHNG BI TON GiI C TRN M HNH PRAM
P-y
NC
NP-y
P
NP
Nu CeP th kt lun P=NP
Nu CeP th kt lun PcNP
P = NP ?
299
Mi bi ton kim chng c th cng gii c
d dng
Cc bi ton ti u t hp u gii c trong thi
gian a thc.
Mi lo lng v s bng n t hp khng cn phi
quan tm
Mt lot cc h m kho cng khai da trn gi
thit P=NP s b v.
Nu P = NP ?
300
Gi s T(n) l hm xc nh trn bin nguyn n. Ta k hiu:
+ T(n)
O(1)
- tp cc hm lu tha ca T(n).
+ (logn)
O(1)
- tp cc hm lu tha ca logarit.
V d: log
2
n, log
5
n e logn
O(1)
, n
2
, n
5
e n
O(1)
v e
2n
, e
5n
e (e
n
)
O(1)
.
Mnh 5.1 (nh tnh ton song song): Lp cc bi ton
gii c trong thi gian T(n)
O(1)
bi m hnh PRAM bng
lp tt c cc bi ton gii c trong khng gian T(n)
O(1)
bi m hnh RAM (my tnh tun t vi b nh ngu nhin)
nu T(n)>logn.
T nh ny suy ra: PRAM c th gii c nhng bi
ton NP-y trong thi gian a thc.
See also: pages 45-48 Parallel computing by Michael J. Quinn
7/17/2010
51
301
1. Khi cc lnh song song:
Parbegin v Parend (Cobegin v Coend): Gi s cc lnh S1,
S2, Sn c thc hin trn n tin trnh ring bit
5.5 CC CU TRC LNH SONG SONG
Parbegin
S1;
S2;
...
Sn;
Parend
Cobegin
S1;
S2;
...
Sn;
Coend
302
2. Cu trc forall (parfor)
Nhiu khi mt s tin trnh (cu lnh) tng t nhau cn phi
bt u thc hin v cng lp li mt s ln. iu ny c th
thc hin c bng cu trc forall
5.5 CC CU TRC LNH SONG SONG
Forall(i=0;i<n; i++){
S1;
S2;
...
Sn;
}
Parend
For(i=0;i<n;i++){
S1;
S2;
...
Sn;
}
In parallel
Hoc
303
1. Thut ton sp xp theo hng
Bi ton: cho mng a[n] cc s nguyn. Hy sp xp tng
dn cc phn t ca mng.
tng thut ton: Trong cch sp xp ny, s nhng s
nh hn mt s chn s c m. S m c s
xc nh v tr ca phn t c chn trong danh sch
cn sp xp. Gi s c n s c lu gi trong mng a[n]
v kt qu sau khi sp xp l b[n].
Thut ton tun t:
5.6 CC THUT TON SP XP SONG SONG
for(i = 0; i < n; i++){
x = o;
for(j = 0; i < n; j++)
if(a[i] > a[j]) x++;
b[x] = a[i];
}
phc tp O(n
2
)
304
Thut ton song song-S dng n b x l
Gi s ta c n b x l. Mi b x l c cp mt s trong
dy s cho trc v n phi tm v tr ca s trong dy
c sp xp.
for(i=0; i<n; i++){ // n b x l thc hin song song
x = o;
for(j=0; i<n; j++)
if(a[i]>a[j]) x++; // m s phn t nh hn
b[x] = a[i]; // Sao sang bng c sp xp
}
Nhn xt: Tt c n b x l thc hin song song nn thut
ton c phc tp l O(n) (tuyn tnh).
(Xemti liu Thut ton song song-S dng n
2
b x l)
305
2. Thut ton sp xp so snh v i ch
dn cc phn t ca mng.
tng thut ton: so snh hai phn t lin k nhau. Nu
cha theo th t th i ch nhau. Qu trnh lp li cho n
khi no khng cn cp no khng tha mn th dng.
Thut ton tun t:
for(i=n-1; i > 0; i--){
for(j=0; j < i; i++){
k = j + 1;
if(a[j] > a[k]){
temp = a[j];
a[j] = a[k];
a[k] = temp;
}
}
}
306
2. Thut ton sp xp so snh v i ch
Thut ton song song:
tng thut ton
Dng n tin trnh kt hp theo nguyn l hnh ng sp
xp mng a[n]
H thng c chia thnh 2 pha: pha chn v pha l
- Pha chn: gm cc tin trnh c nh s chn so snh
vi nhng tin trnh tip theo (s l), nu n gi phn t
lp hn th trao i d liu vi tin trnh .
- Pha l: cc tin trnh c nh s l hot ng tng t
nh trn
7/17/2010
52
307
V d: n=8 v a = (4 , 2 , 7 , 8 , 5 , 1 , 3 , 6)
Pha/TT
P
0
P
1
P
2
P
3
P
4
P
5
P
6
P
7
0 4 2 7 8 5 1 3 6
1 2 4 7 8 1 5 3 6
2 2 4 7 1 8 3 5 6
3 2 4 1 7 3 8 5 6
4 2 1 4 3 7 5 8 6
5 1 2 3 4 5 7 6 8
6 1 2 3 4 5 6 7 8
7 1 2 3 4 5 6 7 8
Sp xp theo nguyn l hnh ng
i ch
Khng i ch
308
Thut ton song song:
Gi s A lu d liu tin trnh l v B lu d liu tin trnh chn
Thut ton song song theo hnh ng c m t theo MPI nh sau:
Pha chn
P
i
, i = 0, 2, ..., n-2 P
i
, i = 1, 3, ..., n-3
recv(&A, P
i+1
); send(&A, P
i-1
);
send(&B, P
i+1
); recv(&B, P
i-1
);
if (A>B) B=A ; if (A>B) A=B ;
Pha l
P
i
, i = 1, 3, ..., n-3 P
i
, i = 0, 2, ..., n-2
send (&A, P
i+1
); recv (&A, P
i-1
);
recv (&B, P
i+1
); send(&B, P
i-1
);
if (A>B) A=B ; if (A>B) B=A ;
309
Kt hp 2 pha ta c thut ton song song:
P
i
, i = 1, 3, ..., n-3 P
i
, i = 0, 2, ..., n-2
send(&A,P
i-1
); recv(&A,P
i+1
);
recv(&B,P
i-1
); send(&B,P
i-1
);
if (A>B) A=B ; if (A>B) B=A ;
if (i<=n-3){ if (i=>2){
send(&A,P
i+1
); recv(&A,P
i-1
);
recv(&B,P
i+1
); send(&B,P
i-1
);
if (A>B) A=B; if (A>B) B=A;
} }
Nhn xt: thut ton c phc tp l O(n).
310
3. Thut ton sp xp MergeSort
dn cc phn t ca mng.
tng thut ton (dng nguyn tc chia tr)
Chia mng a thnh 2 phn
Mi phn li chia i tip cho n khi mi phn nh ch c 1
phn t
Tng cp c ghp li theo th t sp xp
V d
Mng a = (4, 2, 7, 8, 5, 1, 3, 6)
311
6 3 1 5 8 7 2 4
8 7 2 4 6 3 1 5
2 4 8 7 1 5 6 3
4 2 7 8 5 1 3 6
4 2 8 7 5 1 6 3
8 7 4 2 6 5 3 1
8 7 6 5 4 3 2 1
Chia
i
danh
sach
Ghp
kt
hp
312
Chia danh sch thnh tng cp v phn cng cho cc BXL
(4, 2, 7, 8, 5, 1, 3, 6)
P
0
P
0
P
4
P
0
P
2
P
4
P
6
P
0
P
1
P
3
P
5
P
2
P
4
P
6
P
7
P
0
P
2
P
4
P
6
P
0
P
4
P
0
Nhn xt:
thut ton c PT O(nlogn)
7/17/2010
53
313
Xc nh thi gian truyn thng: t
comm
a. Trong giai on phn chia d liu
Truyn d liu Truyn thng ca BXL
t
startup
+ (n/2)t
data
P
0
P
4
t
startup
+ (n/4)t
data
P
0
P
2
; P
4
P
6
t
startup
+ (n/8)t
data
P
0
P
1
; P
2
P
3
; P
4
P
5
; P
6
P
7
b. Trong giai on ghp kt hp c nhng s trao i d liu:
t
startup
+ (n/8)t
data
P
0
P
1
; P
2
P
3
; P
4
P
5
; P
6
P
7
t
startup
+ (n/4)t
data
P
0
P
2
; P
4
P
6
Gi s c p BXL th c hai giai on s thc hin c log p bc.
Vy chi ph truyn thng ca c hai giai on l:
t
comm
= 2(t
startup
+ (n/2)t
data
+ t
startup
+ (n/4)t
data
+t
startup
+ (n/8)t
data
)
= 2(log p)t
startup
+ 2nt
data
Nhc li:
+ t
startup
l thi gian cn thit gi nhng thng ip
khng phi l d liu. N bao gm c thi gian ng gi
thng ip ni gi v thi gian m gi ni nhn.
n gin chng ta gi thit thi gian ny l hng s.
+ t
data
l thi gian cn thit chuyn mt mc d liu
(data item) t ni gi ti ni nhn, c gi thit l hng
s v n l s mc d liu c trao i trong h thng.
314
Vic tnh ton ch thc hin vic ghp cc danh sch con
to danh sch ln hn.
Vic ghp 2 danh sch con c thc hin bng cch
duyt ln lt t u hai danh sch tm phn t nh
hn a vo danh sch mi.
ghp 2 danh sch con c n phn t th cn nhiu nht
2n-1 bc so snh.
c. Xc nh thi gian
tnh ton: t
comp
Nhn xt
315
Xc nh thi gian tnh ton
T
comp
=1 P
0
, P
2
, P
4
, P
6
T
comp
=3 P
0
, P
2
T
comp
=7 P
0
Tng qut,
T
comp
= 1 + 3 + ... + (2i - 1), i = 1, ..., logn
Vy phc tp thi gian tnh ton ca TTSS s dng n
BXL l O(n)
316
3. Thut ton sp xp QuickSort
a. Bi ton: Gi s cn sp xp mt dy con k t v tr start
n end trong danh sch list[ ] vi pilot l v tr ca phn t
trong danh sch c chn lm hoa tiu.
b. tng thut ton
Chia dy sp xp thnh 2 danh sch con sao cho danh sch
1 gm cc phn t nh hn cc phn t ca danh sch 2
Chn mt phn t hoa tiu (pivot) thc hin vic phn chia
Lp li vic phn chia trn cho tng danh sch n khi no
nhng danh sch cui cng ch c 1 phn t
317
c. Thut ton QuickSort tun t c vit qui
QuickSort(list, start, end){
if (start < end){
Partition (list, start, end, pilot);
QuickSort (list, start, pilot);
QuickSort (list, pilot+1, end);
}
}
Trong , Partition() l hm chuyn tt c cc phn t nh
hn hoc bng phn t pilot v trc phn t pilot, nhng
phn t ln hn s c t sau pilot. Thc hin phn tch
xong th pilot l v tr ca phn t hoa tiu danh sch mi
318
d. V d
6 3 1 5 8 7 2 4
4 1 2 3 6 8 7 5
3 1 2 4 5 6 8 7
2 1 3 7 6 8
Danh sch cn sp xp
pilot
Thut ton QuickSort tun t c PT O(nlogn)
Reference: Parallel Programming by Barry Wilkinson
7/17/2010
54
319
e. Song song ha QuickSort
Pht biu li QuickSort
Chn phn t pilot chia dy thnh hai phn: bn tri l
nhng phn t nh hn hoc bng pilot v bn phi l
nhng phn t ln hn pilot .
Phn t pilot c chn vo gia hai dy con c tch v
n l v tr c sp xp sau khi thc hin bc 1.
Bc 1 lp li mt cch qui cho n khi cc dy con ch
cn li mt phn t.
320
Song song ha QuickSort
tng: bt u thc hin mt BXL v chuyn cc li gi
quy cho cc BXL khc.
To ra tp cc tin trnh v t dy cn sp xp vo stack.
Tin trnh u tin (tin trnh ch) thc hin tch dy cho
trc thnh hai dy con v t vo stack.
Nhng tin trnh khc (tin trnh t) x l nhng dy con
c tch ra v thc hin nhng cng vic tng t nh
thut ton nu.
321
P
4
P
0
P
0
P
4
P
0
P
2
P
4
P
6
P
0
P
1
P
6
6 3 1 5 8 7 2 4
4 7 2 3 6 8 7 5
3 1 2 4 5 6 8 7
2 1 3 7 6 8
Danh sch sp xp Phn cng vic cho cc BXL
322
f. nh gi thut ton QuickSort song song
Gi thit rng pilot c chn mt cch l tng mi ln
u tch thnh hai phn c kch c bng nhau.
1. Thi gian tnh ton.
u tin mt b x l phi thao tc vi n s. Sau hai
b x l thao tc trn n/2 s, ri bn b x l thao tc
trn n/4 s, v.v.
Thi gian thc hin thut ton s l:
t
comp
= n + n/2 + n/4 + ~ 2n
2. Thi gian truyn thng.
S trao i d liu gia cc b x l ch xy ra khi thc
hin ghp kt hp ging nh MergeSort.
t
comm
= 2(t
startup
+ (n/2)t
data
+ t
startup
+ (n/4)t
data
+ t
startup
+
(n/8)t
data
)
= 2(log p)t
startup
+ 2nt
data
323
5.7 CC THUT TON SONG SONG THNG DNG
1. Nhn ma trn
M hnh li 2 chiu
M hnh mng tm thu
2. Gii h phng trnh tuyn tnh
3. Tnh biu ca nh
4. T mu th
5. Thut ton Kruskal tm cy khung cc tiu
6. Thut ton tnh tng tin t
7. Phng trnh nhit mt chiu
Bi tp (thamkho ti liu)

Parallel Processing - Xu Ly Song Song PDF

Uploaded by

Document Information

Original Title

Copyright

Available Formats

Share this document

Share or Embed Document

Sharing Options

Did you find this document useful?

Is this content inappropriate?

Copyright:

Available Formats

Parallel Processing - Xu Ly Song Song PDF

Uploaded by

Copyright:

Available Formats

7/17/2010

You might also like