D e p a r tm e n t o f C o m p u te r C o u r s e : La b o r a to r y

Group A

Assignment No: 2(B)

Title of the Assignment: W rite a p ro g ram to im p le m e n t P ar a lle l M e rg e S o rt. U s e

e x is tin g a lg o rith m s a n d m ea s u re th e p e rf o rm a n c e o f s e q u e ntia l a n d p a ra lle l a lg o rithm s .

Objective of the Assignment: S tu d e n ts s h o uld b e a b le to W rite a p ro g ra m to

im p lem en t P a ra lle l M e rg e S o r t a n d c a n m e a s u re th e p e rfo r m a n c e o f s e q u e n tia l a n d

p a ra lle l a lg o r ith m s .

Prerequisite:
1 . B a s ic o f p ro g ra m m in g la n g u a g e

2. C o n c ep t o f M e rg e S o rt

3. C o n c ep t o f P a r alle lis m

Contents for Theory:

1 . W h a t is M e rg e ? U s e o f M e rg e S o rt

2. E x a m p le o f M e rg e s o rt?

3. C o n c ep t o f O p e n M P

4. H o w P a ra lle l M e rg e S o rt W o rk

5. H o w to m e a s u re the p e rfo rm a n c e o f s e q u e n tia l a n d p a ra lle l a lg o r ith m s ?

What is Merge Sort?

M e rg e s o rt is a s o rtin g a lg o r ith m th a t u s e s a d ivid e - a n d - c o n q u e r a p p ro a c h to s o rt a n

a rra y o r a lis t o f e le m e n ts . T h e a lg o rith m w o rk s b y re c u rs ive ly d ivid in g th e in p u t a rra y in to

tw o h a lv e s , s o r tin g e a c h h a lf, a n d th e n m er g in g th e s o rte d h a lve s to p ro d u c e a s o rte d

o u tp u t.

T h e m er g e s o rt a lg o rith m c a n b e b ro k e n d o w n in to th e fo llo w in g s te p s :

1 . D ivid e th e in p u t a rra y in to tw o ha lv e s .

2 . R e c u rs ive ly s o rt th e le ft h a lf o f th e a rra y.

3 . R e c u rs ive ly s o rt th e rig h t h a lf o f th e a rra y.

4 . M e rg e th e tw o s o rte d h a lve s in to a s in g le s o rte d o u tp u t a rra y.

● T h e m e rg in g s te p is w h e re th e b u lk o f th e w o rk h a p p e n s in m e rg e s o rt. T h e

a lg o rith m c o m p a re s th e firs t e le m e n ts o f e a c h s o rte d h alf, s e lec ts th e s m a lle r

M M IT , L o h g a o n , P u n e - 4 7
1
 D e p a r tm e n t o f C o m p u te r C o u r s e : La b o r a to r y

e le m e n t, a n d a p p e n d s it to th e o u tp u t a rra y. T h is p ro c e s s c o n tin u e s u n til a ll

e le m e n ts fro m b o th h a lve s h a v e b e e n a p p e n d e d to th e o u tp u ta rra y.

● T h e tim e c o m p le x ity o f m e rg e s o rt is O ( n lo g n ) , w h ic h m a k e s it an e ffic ie n t s o rtin g

a lg o rith m fo r la rg e in p u t a rra y s . H o w e ve r , m e rg e s o rt a ls o re q u ire s a d d itio n a l

m e m o r y to s to r e th e o u tp u t a rra y , w h ic h c a n m a k e it le s s s u ita b le fo r u s e w ith

lim ite d m e m o ry re s o u rc es .

● In s im p le te rm s , w e c a n s a y th a t th e p ro c e s s o f m e rg e s o rt is to d ivid e th e a r ra y in to

tw o h a lve s , s o rt e a c h h a lf, a nd th en m e rg e th e s o rte d h a lv es b a c k to g e th e r. T h is

p r o c e s s is re p ea te d u n til th e e n tire ar ra y is s o r te d .

● O n e th in g th a t yo u m ig h t w o n d e r is w h a t is th e s p e c ia lty o f th is alg o rith m . W e

a lre a d y h a ve a n u m b e r o f s o rting a lg o rith m s th e n w h y d o w e n e e d th is a lg o rith m ?

O n e o f th e m a in a d v an tag e s o f m e rg e s o rt is th a t it h a s a tim e c o m p le x ity o f O ( n

lo g n ) , w h ic h m e a n s it c a n s o r t la rg e a rra y s r e la tive ly q u ic k ly. It is a ls o a s ta b le s o rt,

w h ic h m e a n s th at th e o r d e r o f e le m e nts w ith e q u a l v a lu e s is p r e s er ve d d u rin g th e

s o r t.

● M e rg e s o rt is a p o p u la r c h o ic e fo r s o rtin g la rg e d a ta s e ts b e c a u s e it is r ela tive ly

e ffic ie n t a nd e a s y to im p le m e n t. It is o fte n u s e d in c o nju n c tio n w ith o th e r

a lg o rith m s , s uc h as q uic k s o rt, to im p ro ve th e o v e ra ll p e rf o rm a n c e o f a s o r tin g

ro utin e .

Example of Merge sort

N o w , le t's s e e th e w o r king o f m e r g e s o r t A lg o r ith m . T o u n d e rs ta n d th e w o rk in g o f th e
m e rg e s o rt

a lg o rith m , le t's ta k e a n u n s o rte d a r ra y. It w ill b e e a s ie r to u n d e rs tan d th e

m e rg e s o rt v ia a n e x a m p le . L e t th e e le m e n ts o f a rra y a re -

● A c c o rd in g to th e m e rg e s o rt, firs t d iv id e th e g iv e n a rra y in to tw o e q u a l h a lv e s .

M e rg e s o rt k e e p s d ivid in g th e lis t in to e q u a l p a rts u n til it c a n n o t b e f u rth e r d ivid e d .

● A s th e re a re e ig h t e le m e n ts in th e g ive n a rr a y, s o it is d iv id e d in to tw o a rra y s o f s ize 4 .

M M IT , L o h g a o n , P u n e - 4 7
2
D e p a r tm e n t o f C o m p u te r C o u r s e : La b o r a to r y

● N o w , a g a in d iv id e th e s e tw o a rra y s in to h a lv e s . A s th e y a re o f s ize 4 , d iv id e th e m

in to n e w a rra ys o f s ize 2.

● N o w , a g a in d iv id e th e s e a rra ys to g e t th e a to m ic v a lu e th a t c a n n o t b e f u rth e r d ivid e d .

● N o w , c o m b in e th e m in th e s a m e m a n n e r th e y w er e b ro k en .

● In c o m b inin g , fir s t c o m p a re th e e le m e nt o f e a c h a rra y a n d th e n c o m b in e the m in to

a n o th e r a rra y in s o rte d o rd e r.

● S o , firs t c o m p a re 1 2 a n d 3 1 , b o th a re in s o r te d p o s itio n s . T h e n c o m p a re 2 5 a n d 8 ,

a n d in th e lis to f tw o v a lu e s , p ut 8 firs t fo llo w e d b y 25 . T h e n c o m p ar e 3 2 a n d 1 7 ,

s o r t th e m a n d p u t 17 f irs t fo llo w e d b y 3 2 . A f te r th a t, c o m p a re 4 0 a n d 4 2 , an d

p la c e th e m s e q u e n tially .

In th e n e x t iter a tio n o f c o m b in in g , n o w c o m p a r e th e a rra ys w ith tw o d a ta va lu e s a n d

m e rg e th e m in to a n ar ra y o f fo u n d v a lu e s in s o rte d o rd e r.

● N o w , th e re is a fin a l m e r g in g o f th e a rra ys . A fte r th e fin a l m e r g in g o f a b o ve a rra ys ,

th e a rra y w illlo o k lik e -

M M IT , L o h g a o n , P u n e - 4 7
3
 D e p a r tm e n t o f C o m p u te r C o u r s e : La b o r a to r y

Concept of OpenMP

● O p e nM P (O p e n M u lti- P ro c e s s in g ) is a n a p p lic a tio n p ro g ra m m ing in te rfa c e ( A P I)

th a t s u p p o rts s h a re d - m e m o ry p a ra lle l p ro g ra m m in g in C , C + + , a n d F o rtra n . It is

used to w rite p ar a lle l p ro g ra m s th a t can ru n on m u ltic o re p ro c e s s o rs ,

m u ltip ro c e s s o r s y s te m s , a n d p a ra llel c o m p u tin g c lu s te rs .

● O p e nM P p ro v id es a s e t o f d ire c tive s a nd fu n c tio n s th a t c a n b e in s e rte d in to th e

s o u rc e c o d e o f a p r o g r a m to p a ra lle liz e its ex e c u tio n . T h e s e d ire c tive s a re s im p le

a n d e as y to u s e , a n d th e y c a n b e a p p lie d to lo o p s , s e c tio n s , f un c tio n s , a n d o th e r

p r o g r am c o n s tru c ts . T h e c o m p ile r th e n g e n e ra te s p a ra lle l c o d e th a t c a n r un o n

m u ltip le p ro c e s s o r s c o n c u rre n tly .

● O p e nM P p ro g ra m s a re d e s ig n e d to ta k e a d v a n ta g e o f th e s h a re d - m e m o ry

a rc hite c tu r e o f m o d e rn p ro c e s s o rs , w h e re m u ltip le p ro c e s s o r c o re s c an a c c e s s

th e s a m e m e m o ry . O p e n M P us e s a fo rk - jo in m o d e l o f p a ra lle l e x e c u tio n , w h e re a

m a s te r th re a d fo rk s m u ltip le w o r k er th re a d s to e x e c u te a p a ra lle l r e g io n o f th e

c o d e , an d th en w a its fo r a ll th re a d s to c o m p le te b e fo re c o n tin u in g w ith th e

s e q u e n tial p a rt o f th e c o d e .

How Parallel Merge Sort Work

● P a ra lle l m e rg e s o rt is a p a ra lle liz e d v e rs io n o f th e m e rg e s o rt alg o rith m th a t ta k e s

a d v a n ta g e o f m u ltip le p ro c e s s o rs o r c o re s to im p ro v e its p e r fo rm a n c e . In p a ra lle l

m e rg e s o r t, th e in p u t a rra y is d iv id ed in to s m a lle r s u b a rra y s , w h ic h a re s o rte d in

p a ra lle l u s in g m u ltip le p ro c e s s o rs o r c o re s . T h e s o rte d s u b a rra ys a re th e n m e rg e d

to g e th e r in p a ra lle l to p ro d u c e th e fin a l s o r te d o u tp u t.

T h e p a ra lle l m e r g e s o r t a lg o rith m c a n b e b ro k e n d o w n in to th e fo llo w in g s te p s :

● D ivid e th e in p u t a rra y in to s m a lle r s u b a rr ay s .

● A s s ig n e a c h s u b a rra y to a s e p a ra te p ro c e s s o r o r c o re fo r s o rting .

● S o rt e a c h s u b a rra y in p a ra lle l u s in g th e m e rg e s o rt a lg o rith m .

● M e rg e th e s o rte d s u b a r ra ys to g e th e r in p a r alle l to p ro d u c e th e fin a l s o rte d o u tp u t.

● T h e m e rg in g s te p in p a ra lle l m er g e s o rt is p e r fo rm e d in a s im ila r w a y to th e

m e rg in g s tep in th e s e q u e n tia l m e rg e s o r t a lg o rith m . H o w e ve r, b e c a u s e th e

s u b a rr ay s a re s o rte d in p a ra lle l, the m e rg in g s te p c a n a ls o b e p e rfo r m e d in

p a ra lle l u s in g m u ltip le p ro c e s s o rs o r c o r es . T h is c an s ig n ific a n tly re d u c e th e tim e

re q u ire d to m e rg e th e s o rte d s u b a rra ys an d p r o d u c e th e fina l o u tp u t.

● P a ra lle l m er g e s o rt c a n p ro v id e s ig n ific a n t p e rf o rm a n c e b e ne f its fo r la rg e in p u t

a rra y s w ith m a n y e le m e n ts , e s p e c ia lly w h e n ru n n in g o n h a rd w a re w ith m u ltip le

p r o c e s s o rs o r c o re s . H o w e ve r, it a ls o re q u ire s a d d itio n a l o v e rh e a d to m a n a g e th e
M M IT , L o h g a o n , P u n e - 4 7
4
 D e p a r tm e n t o f C o m p u te r C o u r s e : La b o r a to r y

p a ra lle liza tio n , a n d m a y n o t a lw ay s p ro v id e p e r fo rm a n c e im p ro v e m e n ts f o r

s m a lle r in p u t s ize s o r w h e n ru n o n h a rd w ar e w ith lim ited p a r alle l p ro c e s s in g

c a p a b ilitie s .

How to measure the performance of sequential and parallel algorithms?

T h e re a re s ev e ra l m e tric s th a t c a n b e u s e d to m e a s u re th e p e rfo r m a n c e o f s e q u e n tia l

a n d p a ra lle l m e rg e s o rt a lg o rith m s :

1 . Execution time: E x e c u tio n tim e is th e a m o un t o f tim e it ta k e s fo r th e alg o rith m to

c o m p le te its s o rtin g o p e r atio n . T his m e tric c a n b e u s e d to c o m p a re th e s p e e d o f

s e q u e n tial a n d p a ra lle l m e rg e s o rt a lg o rith m s .

2 . Speedup: S p e e d u p is th e ra tio o f th e e x e c u tio n tim e o f th e s e q u en tia l m e rg e s o rt

a lg o rith m to th e e x e c u tio n tim e o f th e p a r alle l m e rg e s o rt a lg o rith m . A s p e e d u p o f

g r ea te r th an 1 ind ic ate s tha t th e p a ra lle l a lg o r ith m is fa s te r th a n th e s e q u e n tia l

a lg o rith m .

3 . Efficiency: E ffic ie n c y is th e ra tio o f th e s p e e d u p to the nu m b e r o f p ro c e s s o rs o r

c o re s u s e d in th e p a ra lle l a lg o rith m . T h is m e tric c an b e u s e d to d e te rm in e ho w

w e ll th e p a ra lle l a lg o rith m is u tiliz in g th e a va ila b le r e s o u rc e s .

4 . Scalability : S c a la b ility is th e ab ility o f th e a lg o rith m to m ain ta in its p e rf o rm a n c e a s

th e inp u t s ize a n d n u m b e r o f p ro c e s s o rs o r c o re s in c re a s e . A s c ala b le alg o rith m

w ill m a in ta in a c o ns is te n t spe e du p a nd e fficie nc y a s m o re re sou rc es a re ad de d.

T o m e a s u re th e p e r fo rm a n c e o f s e q ue n tia l an d p a ra lle l m e rg e s o rt a lg o rith m s , y o u c a n

p e rf o rm

e x p e rim e n ts o n d iff e re n t in p ut s iz e s a n d n u m b e rs o f p ro c e s s o r s o r c o re s . B y

m e a s u rin g th e e x e c u tio n tim e , s p e e d u p , e ffic ie n c y, a n d s c a la b ility o f th e

a lg o rith m s u n d e r d iffe re n t c o n d itio n s , y o u c an d e te rm in e w h ic h a lg o rithm is m o re

e ffic ien t fo r d iffe re n t in p u t s ize s a n d h a rd w a re c o n fig ur a tio n s . A d d itio n ally , y o u c a n

u s e p r o filing to o ls to a n a lyz e th e p e rf o rm a n c e o f th e a lg o rith m s a n d id e n tify ar e as

fo r o p tim iza tio n

Conclusion- In th is w a y w e c a n im p le m e n t M er g e S o rt in p a ra lle l w a y u s in g
O p e nM P a ls o c o m e to k n o w h o w to h o w to m e a s u re p e rf o rm a n c e o f s e r ia l a n d

p a ra lle l a lg o r ith m

FAQs:

1. W h a t is p a ra lle l M e rg e S o rt?

2 . H o w d o e s P a ra lle l M er g e S o rt w o rk ?

M M IT , L o h g a o n , P u n e - 4 7
5
D e p a r tm e n t o f C o m p u te r C o u r s e : La b o r a to r y

3 . H o w d o yo u im p le m e n t P a ra lle l M e r g e S o rt u s in g O p e n M P ?

4 . W h a t a re th e a d v a n ta g e s o f P a ra lle l M e rg e S o rt?

5 . D iffe re n c e b e tw ee n s e ria l M e rg e s o rt a n d p a ra lle l M e rg e s o rt

Reference link
● h ttps:/ /w w w .ge e ksforge ek s.org/m e rge -so rt/

● h ttps:/ /w w w .jav a tpo int.c om / m e rge -sort

M M IT , L o h g a o n , P u n e - 4 7
6