Levels of Parallel Processing
Instruction Level Parallelism(ILP)
• Multiple Instructions at the same time
• Multiple Processing Elements
• Pipeline Processing
• Very Large Instruction Word Computers(VLIW)
Loop Level Parallelism
• Unfold the loop/ Loop unrolling
• Execute multiple iterations at the same time
• For(i=0; i<10; i++)
– A[i]=b[i]+c[i]
• For(i=0; i<10; i++)
sum=sum+a[i]
Procedure Level
• Functions/ Threads
• Independent
• Sum(a,b)
• Mul(c,d)
Program Level
• Multiple programs at the same time
• Independent
• Managed by Operating System
Parallel Processing Interconnect
Parallel Processing Interconnect
Shared Memory System
• Common Memory
• All processors access the memory using same
bus
• Balanced access
• Symmetric Muptiprocessor
Message Passing Systems
• Distributed Memory, Not Global
• Each processor having access to its local memory
• Data transfer using message passing
• Send and receive
• Programmer need to take care of data sharing
Interconnection Network
Interconnection Networks
• Mode of operation
– Synchronous: All components use the same clock
• Lock-Step
– Asynchronous
• Global clock is not required
• Handshaking
• Faster
Control Strategy
• Centralized
– One control system for all the components
• Decentralized
– Every component is having its own control system
Switching Technique
• Circuit Switching
– Establish the path first
– Path should remain
– Messages are transferred via path
• Packet Switching
– Message divided in to small units: Packet
– Packets are sent over the network
– Store-and-forward mechanism
Topology
• The way one unit(Processor & Memory)
communicates with other unit
• Mapping function
• Fully connected
• Ring
“Interconnection networks can be
classified as static and dynamic”
Static: Fixed
Dynamic: as per the need
Static IN
Dynamic IN(Single and Multilevel)
Dynamic IN(Crossbar Switch)
Shared Memory and IN
• Bus based
– All processors communicate via same bus
– What if all processors are trying to communicate?
