Professional Documents
Culture Documents
University
CpE 440
Computer Architecture
Dr. Haithem Al-Mefleh
Computer Engineering Department
Yarmouk University, Second 2020-2021
Multicores, Multiprocessors,
and Clusters
1
CpE 440, Second 2020-2021, Yarmouk 2/26/2021
University
• Parallel programming
• Execute efficiently in performance and power
2
CpE 440, Second 2020-2021, Yarmouk 2/26/2021
University
3
CpE 440, Second 2020-2021, Yarmouk 2/26/2021
University
• Time Synchronization
• Scheduling
0.1%; 0.001
4
CpE 440, Second 2020-2021, Yarmouk 2/26/2021
University
10
10
5
CpE 440, Second 2020-2021, Yarmouk 2/26/2021
University
Time = max(200t, 9800t/99) + 10t = 210t Time = max(500t, 9500t/99) + 10t = 510t
Speedup = 48 Speedup = 20
11
11
12
12
6
CpE 440, Second 2020-2021, Yarmouk 2/26/2021
University
13
13
2 styles of SMP,…
• UMA – Uniform Memory Access
• the same time to access main memory – any processor requests it
and any word is requested
14
14
7
CpE 440, Second 2020-2021, Yarmouk 2/26/2021
University
Synchronization
• Processors should coordinate when sharing data
• Lock is one mechanism – one processor access a shared data at a
time
15
15
16
16
8
CpE 440, Second 2020-2021, Yarmouk 2/26/2021
University
17
17
18
18
9
CpE 440, Second 2020-2021, Yarmouk 2/26/2021
University
• Each processor –
private physical space
• Communicate with
message passing
• ACK is possible
19
19
• Disadvantages –
• cost of administration of n machines
cost of administration of n independent
machines
• cost of administration shared M with n processors cost of
administration of 1 machine
20
20
10
CpE 440, Second 2020-2021, Yarmouk 2/26/2021
University
21
21
22
22
11
CpE 440, Second 2020-2021, Yarmouk 2/26/2021
University
23
23
• Better availability
• Much easier to disconnect a machine, reinstall, replace, …
24
24
12
CpE 440, Second 2020-2021, Yarmouk 2/26/2021
University
Hardware Multithreading
25
25
• 2 approaches
- Interleaving
- Individual threads
- start-up overhead
26
26
13
CpE 440, Second 2020-2021, Yarmouk 2/26/2021
University
27
Example
28
28
14
CpE 440, Second 2020-2021, Yarmouk 2/26/2021
University
29
29
SISD – a uniprocessor
MIMD – a multiprocessor
• Different programs
• 1 program – conditional statements
• SPMD (Single Program Multiple Data)
30
30
15
CpE 440, Second 2020-2021, Yarmouk 2/26/2021
University
31
31
32
32
16
CpE 440, Second 2020-2021, Yarmouk 2/26/2021
University
Vector
• Pipelined ALU
• Get data into registers, operate on them sequentially, store result
back to M
• Vector registers
• Like an entire loop
• Hardware doesn’t have to check for data hazards in the same Vector
• Control hazards in loops are nonexistent
• Number of elements in a separate register
33
33
Introduction to Graphics
Processing Units (GPU)
34
34
17
CpE 440, Second 2020-2021, Yarmouk 2/26/2021
University
35
35
36
18
CpE 440, Second 2020-2021, Yarmouk 2/26/2021
University
37
37
• Heterogenous/Identical
38
38
19
CpE 440, Second 2020-2021, Yarmouk 2/26/2021
University
39
39
Introduction to Multiprocessor
Network Topologies
Multicore chips networks on chips to connect cores
40
40
20
CpE 440, Second 2020-2021, Yarmouk 2/26/2021
University
• Cost depends on
• Switches
• Links on a switch
• Width (bits #) per link
• Length of links
• Performance
• Throughput – max # of messages in a time
• Latency to send and receive messages
• Contention
• …
• Fault tolerance
• Power Efficiency
41
41
• Links – bidirectional,…
• Processor-Memory Node
• Bus
• Total BW = BW of the bus
= 2xBWlink
• the bisection bandwidth = BWlink
42
42
21
CpE 440, Second 2020-2021, Yarmouk 2/26/2021
University
• Ring
• Total BW = PxBWlink
• the bisection bandwidth = 2xBWlink
43
43
• Fully Connected
• Each P – a bidirectional link to every other P
• Total BW = P × (P - 1)/2
• the bisection bandwidth is (P/2)2
44
44
22
CpE 440, Second 2020-2021, Yarmouk 2/26/2021
University
45
45
46
46
23
CpE 440, Second 2020-2021, Yarmouk 2/26/2021
University
47
47
48
48
24
CpE 440, Second 2020-2021, Yarmouk 2/26/2021
University
Any questions/comments?
Thank you
49
49
25