You are on page 1of 15

An Overview of Myrinet

By:
Ralph Zajac
What is Myrinet?
LAN designed for clusters
Based on USCDs ATOMIC LAN
Has many characteristics of MPP
message-passing networks
MPP Network
Characteristics
High data rate
Regular topology and scalability
Very low error rate
Cut-through routing
Flow control
ATOMIC LAN: Overview
Based on technology from
Caltechs Mosaic project
Used TCP/IP at burst rates of 400
Mb/s
Error rate less than 10-15
ATOMIC LAN: Limitations
Asynchronous communication could lead
to deadlock
Network topology
Complex
Hidden computing power
Performance limited by lack of DMA in
host interface
End-to-end data rates limited by TCP/IP
stack of OS
Myrinet
Designed to address the limitations
of ATOMIC LAN
Not limited to an existing MPP
network
Specifications published in
ANSI/VITA 26-1998
Myrinet: Links
Can be copper
(up to 25m) or STOP
optical-fiber
GO
Full duplex
Uses non-return-
to-zero encoding
Slack Buffer
Flow control run
by a slack buffer
Myrinet: Packets and
Routing
Leading byte determines the
outgoing port
MSB of each header byte
distinguishes between host and
switch packets
Variable payload length
8-bit CRC
Myrinet: Switches
Use blocking-cut-through routing
Each contains two custom VLSI
chips
crossbar-switch
dual interface
Topology is arbitrary
Myrinet: Host Interface
Available for PCI and Sbus
32-bit SRAM for MCP
Can act as a slave or a bus master
Myrinet: Software
MCP
Handles all interfacing with the
network
Continuously mapping and monitoring
the network
Host Software
TCP/IP and UDP/IP
Myrinet API
High Performance Virtual
Machine
Windows NT 4.0 cluster built at UCSD and
UIUC for the NCSA
96 dual processor nodes
Used a fat-tree network with 8-port
Myrinet switches
Was a 56 GFlops system with:
45 GB of DRAM
1.6 MB/s bisection bandwidth
400 GB of disk storage
HPVM: Testing
Tested on four programs
ZeusMP: astrophysical phenomena in 3D
Cactus: modular 3D numerical relativity
AS-PCG kernel: linear systems solver
QMC kernel: Quantum Monte Carlo
simulator
Compared with a Cray T3E and an
Origin 2000
HPVM: Results
Performed from 2 to 4 times slower
Slower floating point calculations
Flow control limitations of Myrinet
Much better price/performance
ratio
Conclusions
Myrinet is:
Flexible
Fast
Affordable systems with
acceptable performance